GPS and the misleading average
Marcin Polkowski
Translated from Polish. Originally published on morony.pl on 13 Jul 2010.
I use a Garmin handheld GPS receiver, the GPSmap60CSx. It's a very solid receiver, which only got a successor in 2010. The device has a decent antenna that lets it reach the advertised position accuracy of 3 meters. That kind of accuracy is excellent for hiking and navigation. The question came up whether such a device can achieve survey-grade accuracy (maybe not to the millimeter, but centimeters would be satisfying too).
For the experiment I used an external Garmin antenna, the GA-30. I mounted the antenna on a one-meter mast on the roof of a building, so that it had the best possible view of the sky. The GPS constantly showed 12 satellites in view (the receiver has 12 channels) and reported an accuracy of 2 meters.
The receiver itself has a function for saving a waypoint with averaging, which increases the accuracy as the number of measurements grows. I connected the device to a computer over USB and set up two parallel measurements. The computer collected coordinates from the receiver at a rate of several readings per second. In parallel, the receiver was averaging the coordinates.
Of course, the antenna stayed completely still for the whole duration of the experiment.
I collected measurements for almost 3 days without a break. The receiver passed coordinates to the computer with a (recording) precision of 0.000001 degrees. It displayed the average it computed with a precision of 0.00001 degrees.
At the end, I first compared the averaged position from the receiver with the position obtained by averaging all the measurements from the computer. The results turned out to be in complete agreement (within the receiver's recording precision).
The problem came up when analyzing the error of the resulting value. When computing an arithmetic mean, the error (uncertainty) is the standard deviation, which for the measurements from the computer came to about 3 meters. For the same data, the receiver reported an accuracy of 0.1 meters! My first thought was that the receiver had a better algorithm for calculating the error than a plain standard deviation, one that follows from the specifics of the measurement. I searched the web for a bit and found a few formulas (I won't quote them, because I don't know which one is correct) that took the measurement time into account – the longer the time, the better the accuracy. That didn't convince me, so I turned to a graphical analysis.
Below are 4 views of a three-dimensional histogram of the measurements. The size of each bar equals the recording precision of a measurement, i.e. 0.000001 by 0.000001 degrees. The horizontal axes show the distance from the mean in meters. The first image shows a significant concentration of measurements around a certain point – a good sign. Of course, you can also see that there were excursions as far as 30 meters from the center.
The problem is visible in images 2, 3 and 4 – the mean is shifted relative to the center of the peak.




The two figures above show side views of the 3D histogram, so you only see the tallest bar for a given value. The actual two-dimensional histograms for latitude and longitude are below:


Conclusion?
The conclusion is that you can't determine a position more accurately than a few meters with a GPS receiver.
The second conclusion is that Garmin cheats when averaging.