#distance#haversine#coordinates#great circle#bearing

How to Calculate the Distance Between Two Coordinates (Haversine vs Vincenty, With Worked Examples)

Calculate the distance between two GPS coordinates with the haversine formula, see how far it drifts from the exact WGS84 ellipsoid answer, and avoid the most common mistakes. Includes a Seoul–Busan worked example.

by 9 min readLast reviewed

You have two GPS coordinates — how far apart are they? Most mapping apps answer with the haversine formula, which computes the great-circle distance over a spherical Earth. But the Earth is not a perfect sphere, and for surveying or long-range work the more exact ellipsoidal methods (Vincenty, Karney) matter. This guide walks through the formula, a worked example you can check yourself, and real numbers showing how big the difference actually is.

Straight Line on a Sphere: the Great Circle

On a flat map, the shortest path between two points is a straight line. On a sphere, it is an arc of a great circle — the circle you get by slicing the sphere through its center and both points. This is why long-haul flights between, say, Paris and Tokyo arc far to the north instead of following a straight line on a Mercator map: the "curved" route is actually the shortest one.

On the real Earth, which is slightly flattened at the poles, the shortest path is called a geodesic. It is very close to a great circle, but not identical.

The Haversine Formula

Given two points with latitude φ and longitude λ (converted to radians):

a = sin²(Δφ/2) + cos φ₁ · cos φ₂ · sin²(Δλ/2)
d = 2R · asin(√a)

where R is the Earth's mean radius, 6,371,008.8 meters. That value is the mean of the three semi-axes of the WGS84 ellipsoid, (2a + b) / 3, using a = 6,378,137 m and b ≈ 6,356,752.3 m. Haversine is the standard choice over the simpler spherical law of cosines because it stays numerically well-conditioned even when the two points are very close together.

Worked Example: Seoul to Busan

  • Seoul: 37.5665, 126.9780
  • Busan: 35.1796, 129.0756

Step by step:

  1. φ₁ = 37.5665° → 0.65566 rad, φ₂ = 35.1796° → 0.61400 rad
  2. Δφ = 35.1796 − 37.5665 = −2.3869° → −0.041659 rad
  3. Δλ = 129.0756 − 126.9780 = 2.0976° → 0.036610 rad
  4. a = sin²(−0.020830) + cos(0.65566) · cos(0.61400) · sin²(0.018305) ≈ 0.0006509
  5. d = 2 × 6,371,008.8 × asin(√0.0006509) ≈ 325.11 km

Now compare that with the ellipsoidal answer on WGS84 (computed with GeographicLib, Karney's algorithm): 324.92 km. The sphere overestimates this route by about 196 m, or 0.06%. For a road trip that is nothing. For a survey baseline it is enormous.

The straight-line figure is also only the floor for travel. Roads, rail lines, shipping lanes and air corridors all add distance, so treat great-circle distance as "as the crow flies", never as a driving estimate.

Initial Bearing: Which Way to Go

The same two points also define an initial bearing — the compass direction you face at the start:

θ = atan2( sin Δλ · cos φ₂ , cos φ₁ · sin φ₂ − sin φ₁ · cos φ₂ · cos Δλ )

For Seoul → Busan this gives 144.09° on the sphere (southeast); the ellipsoidal azimuth is 143.97°. Note the word initial: on long routes the bearing changes continuously along the great circle. Paris → Los Angeles starts at about 314° — northwest — even though Los Angeles lies southwest of Paris on a flat map.

How Big Is the Haversine Error, Really?

The spherical model is wrong in a predictable way: the Earth bulges at the equator, so east–west distances near the equator are longer than a sphere predicts, and north–south distances are a little shorter. I compared haversine (R = 6,371,008.8 m) against the WGS84 geodesic for several routes:

RouteHaversineWGS84 geodesicDifference
Seoul → Busan325.112 km324.915 km+196 m (+0.06%)
Paris → Tokyo9,711.7 km9,735.3 km−23.6 km (−0.24%)
Paris → Los Angeles9,085.5 km9,107.1 km−21.6 km (−0.24%)
Equator, 0° → 90° E10,007.6 km10,018.8 km−11.2 km (−0.11%)
Equator → North Pole10,007.6 km10,002.0 km+5.6 km (+0.06%)

Across 200,000 random point pairs spread evenly over the globe, the haversine error ranged from −0.44% to +0.56%, with a median absolute error of 0.11% and 95% of pairs within 0.36%. That matches the commonly quoted rule that a spherical model gives errors "typically up to 0.3%".

The percentage does not shrink for short distances. A 1 km line at 37.5° N comes out as:

  • 1,001.9 m with haversine when it runs north–south (+1.9 m)
  • 999.8 m when it runs northeast (−0.2 m)
  • 997.6 m when it runs east–west (−2.4 m)

A couple of meters per kilometer sounds tiny, but it is larger than the error of an RTK-corrected GNSS position, which is typically at the centimeter level.

When Haversine Is Fine and When You Need Vincenty or Karney

Haversine is fine for: "how far is it" questions, travel and logistics estimates, nearest-store searches, fitness tracking, geofencing with generous radii, and anything where a few tenths of a percent is invisible next to other errors (consumer GPS, route detours).

Use an ellipsoidal method for: surveying and cadastral work, engineering baselines, comparing your result against official geodetic values, aviation and maritime planning to published tolerances, and any workflow where your positions are already better than the sphere's error.

I build drone ground control software, and before planning image-mapping missions I measured distances and areas of the target site. For drone position errors I relied on RTK/PPK corrections, which reach centimetre-level positions — and at that level, a sphere-based distance (off by up to ~0.5%, i.e. metres per kilometre) can be larger than the positioning error itself. That is the point at which a geodesic library earns its place.

Among ellipsoidal methods, Vincenty's formulae (1975) are widely used and very accurate: for Seoul → Busan and Paris → Tokyo my Vincenty implementation agreed with Karney's algorithm to within 0.01 mm. Their known weakness is nearly antipodal points (two points almost on opposite sides of the Earth), where the iteration fails to converge — in my tests, (0°, 0°) → (0.5° N, 179.7° E) never converged, while Karney's method returned 19,944.1 km. Karney's algorithm (2013), implemented in GeographicLib and used by PROJ/pyproj, converges for all point pairs and is accurate to roughly machine precision, so it is the safer default today.

Common Mistakes

  1. Passing degrees to trig functions. sin() and cos() in almost every language expect radians. Forgetting the conversion gives nonsense with no error message.
  2. Swapping latitude and longitude. Many formats (GeoJSON, WKT, most GIS libraries in x/y order) put longitude first. Swap Seoul and Busan's values and you get 284.8 km instead of 325.1 km — wrong, but plausible enough to slip through.
  3. Using an inconsistent radius. Using the equatorial radius (6,378,137 m) gives 325.48 km for Seoul–Busan; the polar radius gives 324.38 km. Pick the mean radius for haversine and keep it consistent.
  4. Mixing datums. Haversine and Vincenty both assume both points are in the same geographic system. A WGS84 point and a point in an older local datum can differ by tens or hundreds of meters before you calculate anything. Convert first with the EPSG converter.
  5. Measuring with projected coordinates as if they were flat everywhere. Pythagoras on UTM eastings and northings works well within one zone, but not across zone boundaries. The UTM converter shows which zone each point falls in.
  6. Ignoring altitude. Both formulas give distance along the ellipsoid surface. A drone 120 m above its takeoff point is farther away in 3D than the horizontal figure suggests — and GPS altitude itself can disagree with barometric or map heights, something I have seen firsthand in drone operations.
  7. Rounding coordinates too early. Each decimal place of a degree matters: 0.0001° of latitude is about 11 m. Keep at least 5–6 decimals until the final result.

FAQ

Is Google Maps' "measure distance" haversine or ellipsoidal? Google does not publish its exact method, so I won't guess. For short distances the two approaches differ by about a meter per kilometer or less, so either will look the same on screen; for exact work, compute it yourself with a geodesic library.

How accurate is haversine for short distances, like under 1 km? The relative error stays in the same range, roughly 0.1–0.5% depending on latitude and direction — about 0.2 to 2.4 m over 1 km at mid-latitudes in my tests. That is fine for navigation but not for survey-grade work.

What should I use in code? For quick estimates, the haversine formula above. For accuracy, use a maintained geodesic library rather than hand-rolling Vincenty: GeographicLib (Geodesic.WGS84.Inverse(lat1, lon1, lat2, lon2) in Python) or pyproj (Geod(ellps="WGS84").inv(lon1, lat1, lon2, lat2) — note longitude first).

Skip the Math

If you just need the number, the free distance calculator does this in one click: click two points on the map (or type coordinates or addresses) and get the haversine great-circle distance in kilometers, miles, or nautical miles plus the initial bearing. Need the area of a field or survey site instead? Use the area calculator. To convert coordinate formats first (DMS, decimal degrees), use the coordinate converter.

Sources

All distances in this post were computed with Python using GeographicLib 2.1 and pyproj 3.8.0 on the WGS84 ellipsoid, and haversine with R = 6,371,008.8 m.

Related Posts

#geocoding#reverse geocoding#coordinates

Geocoding vs Reverse Geocoding: How Addresses Become Coordinates (and Back)

What geocoding and reverse geocoding actually do, how match quality levels work (rooftop vs interpolated), why results differ between providers, and what accuracy you can expect.

8 min read
Read More
#MGRS#UTM#coordinates

MGRS vs UTM: How the Military Grid Reference System Builds on UTM

MGRS and UTM share the same math but read differently. Learn how MGRS compresses UTM coordinates into grid references like 4QFJ 12345 67890, and when each system is the right choice.

8 min read
Read More
#UTM#coordinates#GIS

What Is the UTM Coordinate System? Zones, Easting, and Northing Explained

Understand the UTM (Universal Transverse Mercator) coordinate system: how its 60 zones work, what easting and northing mean, and when to use UTM instead of latitude/longitude.

9 min read
Read More