GNSS Test Metrology
Record and Replay or Record and Playback, a testing technique adapted to geolocation in constrained environments
GNSS tests specifications and methodologies
The field of geolocation has developed an essential reference methodology for evaluating and characterizing GNSS terminals intended for land navigation.
The EN16803 series of standards aims to inform user choices and simplify the integration of satellite technologies with embedded systems. This methodology is based on a metrology technique identified by the term “Record & Replay” and offers several key advantages:
- Restore test environments that are both “reproducible” and “representative” of real-world conditions;
- Dissociate “Trueness” and “Precision” errors so as to improve the analyses;
- stablish reliable baseline scenarios for comparing measurement outcomes.
Geolocation errors, degraded signal and environmental masking
In a perfect world, the positions calculated by trilateration using the signals transmitted by the satellites would always be accurate to within a few centimetres. Unfortunately, in addition to the intrinsic quality of the receivers, many factors alter the measurements made by a GNSS receiver and degrade the final
geolocation data.
To begin with, the GNSS system itself suffers from multiple imperfections including so-called “Global” errors. For this reason, the satellite navigation system is complemented with the broadcasting of assistance messages to increase the performance of receivers compatible with SBAS systems, such as EGNOS for the European continent.
In addition, for terrestrial applications, the satellite signals are affected by several phenomena caused by the immediate surrounding of the receiving antenna. These are the so-called “Local” errors, such as terrain, bridges, infrastructures, vegetation and interference of any type. Depending on the areas covered, the trajectories calculated by the terminals deviate more or less from that actually taken by the vehicle (antenna), i.e. the “reference trajectory”, also called “ground truth”.
Figure 1 – Sources of error in urban geolocation
- Orbits & Clocks
- Satellite geometries
- Ionosphere, troposphere
LOCAL ERRORS
- Obstruction, Attenuation
- Multipath & Diffraction
- Interference, Jamming,…
- Receiving chain
- Algorithms & Services
- Navigation sensors
CLASSIFICATION POSITION errors
To study those phenomena having the greatest impact and likely to be the most frequent, the different types of errors are displayed as a risk matrix. As the “Global” errors can be considered to be handled by the regional SBAS system, the pre-eminence of the so-called “Local” errors should be addressed. In addition, they add to the Global errors and the intrinsic inaccuracies induced by the GNSS terminal.
Description of the main sources of local errors
To observe the effects of local phenomena on the propagation of signals, a dozen identical receivers – with the same configuration and sharing the same antenna – on board a vehicle, were driven through urban and peri-urban areas.
We focus on four particularly impacting phenomena to visualize the trajectories calculated by the receivers.
Positioning errors due to bridges
In the picture, below, the test vehicle passes under a bridge in both directions. In both cases, the trajectories diverge under the bridge and converge further on. Here it is easy to understand the
shortcomings of results based on a single pass, in other words based on a single measurement.
Figure 3 – Effect of alteration of GNSS signals on receivers passing under a bridge.
Positioning errors due to buildings
In the composite image (in order to show the main building) below, all the receiver trajectories are deviated towards the building alongside the avenue. The situation highlights the consequences of a phenomenon called “multipath”. When a receiver captures reflected waves, the signal propagation time – used to calculate the pseudo-distances – is increased and the accuracy of the end position is degraded. This effect is well known and easily observable during static measurements.
Figure 4 – Effects of GNSS signalpropagation on receivers near a building.
Positioning errors due to vegetation
In the image below, the test vehicle is on an avenue lined by trees whose branches and canopy cover the road. The foliage attenuates, and more importantly, diffracts the radio waves arrivingfrom the satellites, thus degrading signal reception. This results in dispersed trajectories. Each receiver provides a different measurement. Note that due to the proximity of buildings, the centre of the position distribution, in the presence of multipath, deviates slightly from the reference trajectory.
Figure 5 – Effect of diffraction of GNSS signals on receivers passing under foliage showing, superimposed, the position distribution.
Positioning errors due to interferences
In the image below, the on-board receivers have been disturbed by “transitory” interference. On the outward journey, twenty minutes earlier, no problem had been detected for the trajectories on the other side of the expressway.
On the return journey, this unidentified interference degrades the accuracy of the receivers with a visible dispersion of the trajectories. In other situations, intentional or unintentional interference could completely block out the GNSS band preventing any position measurement.
NB. : In this case, the source of the interference seems to come from the bottom right, guided by the two lateral buildings.
Figure 6 – Effect of unidentified temporary interference on signals for GNSS receiver.
Trueness and precision of position measurements
Receivers of the same batch behave differently depending on the environment. For a predominantly multipath situation, they all converge to the same wrong position. On the other hand, when the propagation phenomena become more complex with multiple diffractions, such as a reception under foliage, all the receivers produce positions with different errors. For complex environments, we have a combination of these two behaviours.
The first behaviour is deterministic. Metrology vocabulary uses the term measurement “Trueness” for: “closeness of agreement between the average of an infinite number of replicate measured values and a reference value”.
The second behaviour is NON-deterministic. Metrology then uses the term measurement “Precision” for: “closeness of agreement between indications or measured values obtained by replicate measurements on the same or similar objects under specified conditions “.
Terrestrial applications often offer a varied mix of environments where “Trueness” and “Precision” errors accumulate. It is essential to consider both components in order to characterize and study GNSS receiver performance.
Static distribution of the different positioning errors:
Figure 7 – Combination of deterministic and non-deterministic errors.
Figure 8 – A single position measurement at a point has 2 unknowns: The weight of deterministic (Trueness) errors compared to those that are not; The spread of non deterministic errors (Precision).
Figure 9 – Statistical distributions of errors for a trajectory (scenario), i.e. the percentage of all errors (probability) lying beneath a given accuracy level.
Above, 95% of the positions calculated during a replayhave an accuracy better than 1.5m; this same value is only reached with ~ 80% of the positions calculated during another replay – see vertical line [d]. The horizontal line [e] illustrates the spread of the horizontal position by considering 95% of the positions of two replays: for one the displayed accuracy is ~ 1.5m and for the other it is degraded to 3.5m. This curve will always point to the same reference points [a], [b] and [c] recommended by the
standard EN16803-1 and corresponds to the percentage of measurements respectively less than 50%, 75% and 95%
determines the degree of dispersion of the “random” component of the measurements. Thus, in certain environments, each additional receiver will produce a different result. However, the analyses of a single on-site campaign rely on just one single sample (single trajectory of the terminal under test), where a
panel of measurements is essential. In fact, the available statistics prove insufficient to characterize a receiver, even at the cost of doing long runs.
Live testing is therefore rather intended for final integration.
Figure 10 – Visualization of the combined deterministic and non-deterministic errors
for a street lined with trees on one side and houses on the other.
On the other hand:
A constellation generator will synthesize ideal signals derived from mathematical models, and in any case, not representative of the real environment. The measurements will then only be deterministic, i.e. subject to “systematic” errors. Repeated simulations on the same receiver will always produce the same measurements. Nevertheless, this type of test bench offers many advantages for simulating unobservable situations in the real world.
Disparities in analysis capabilities on position errors based on :
Figure 11 – Typical results for repeated measurements obtained, respectively from left to right, with synthetic signals and real-world signals.
Each situation combines both Trueness and Precision errors. This latter component requires several runs in the same configuration to determine the potential measurement spread.
Figure 13 – Position error profiles (measured trajectories / DUT) depending on the environment.
What is GNSS metrology?
As a first approach, characterization of GNSS performance would require many receivers on the same test vehicle. This method is certainly useful in the experimental stage, especially to understand the impact of propagation phenomena on positioning errors. However, it has major disadvantages, both from a logistics point of view and because of the basic metrological requirements.
To obtain reliable and useful measurements, from an operational point of view, the tests must be “representative” of the areas to be covered and “reproducible” to check the results and make valid comparisons, for example, between 2 terminals, 2 firmwares, 2 settings, 2 antennas, even between 2 hybridizations.
Under these conditions, replay techniques, often referred to as “Record and Replay”, meet the expected requirements. For the record, this metrology method consists in digitizing the GNSS signals received by the antenna on board the definition vehicle, taking care to collect all the data associated with the tests (VIDEO, INS, DMI, NRTK, …), above all, the ground truth. Thus at the end of the campaign the GNSS
signals and other data are synchronized and restored on a replay bench consisting of an “SDR replayer”.
Replaying the same scenario on a receiver makes it possible to reproduce the recording conditions identically. Each pass generates new measurements, equivalent to using an additional unit, virtually onboard. Compiling the results thus highlights the non-deterministic errors, i.e. those which by their random nature emerge from the others.
Test laboratories such as GNSS GUIDE design and market test data directly replayable on the main simulation instruments capable of operating in two modes: Simulation and Replay. The replay configurations are generally much more affordable than the larger, structurally more complex constellation generators. In addition, the implementation of replay sessions is simple, fast and requires no special training.
As well as scenarios made on request, the available libraries already cover a multitude of cases, previously inaccessible for an isolated user. They open up the possibility of testing terminals in different latitudes with varied terrain and neighbourhoods composed of typical architectures.
Learn from positioning errors
The French Space Agency (CNES) has financed several R & D contracts for the development and validation of this replay technique (Record & Replay). It is already recommended by CEN / CENELEC through the series of EN16803 standards to characterize and classify the performance of GNSS terminals. This methodology complies with the basic principles of metrology. The test conditions are reproducible and representative of operational conditions. The measurements are repeatable and allow separating the systematic errors (Trueness) from the random errors (Precision). Measurement uncertainties are also accurately established.
During an on-site measurement campaign, the statistical distributions of two identical receivers on board the same vehicle lead to different results. Thus, no characterization can be established at this stage.
With a replay bench, after several iterations of the same scenario, the average values of the measurements on a CDF tend towards a curve characterizing the performance for that scenario.
Instrumentation dedicated to replay operations is less complex and less expensive. Statistical models of simulations are replaced by scenarios of GNSS signals previously digitized in the field or on constellation generators. Thus, whether they come from a real or synthetic environment, these GNSS signals are easily restored, while drastically reducing the preparation and execution times. The economic benefits of this test technique are now evident and are favouring its adoption by the transportation industries.