Understanding Covariance Quality in Robot Localisation

(Written by Anthony Glynn, Kudan CTO)

Consider a robot navigating the bustling aisles of a warehouse, swiftly picking up and delivering items. It must decide how quickly to move and how much space to leave when turning corners to avoid accidents, like clipping a shelf and causing a cascade of items. For this, the robot relies on its localisation module which integrates data from its sensors, such as cameras, lidars and wheel odometry, and combines this with a prebuilt map of the environment to pinpoint its precise location. The localisation system must not only output its position but also assess how confident it is in its estimate. This confidence, quantified by something called covariance, is crucial. Accurate location data is essential, but so is the robot’s certainty about this data. If the robot misjudges its certainty, being either too confident or too cautious, it could lead to reckless behaviour or to an overly hesitant and inefficient system.

Covariance

Rather than relying on a single, precise location estimate, our localisation system instead outputs an entire probability distribution. Covariance, which comes from modelling our estimate as a Gaussian distribution, extends the concept of variance to multiple dimensions. It is represented as a matrix and captures both the notion of how spread out our estimates are, as well as the correlation between the different aspects of the robot’s pose such as the x and y coordinates. A larger covariance indicates a wider spread, signalling greater uncertainty: the robot’s true location could fall within a broader range of values.

(Image: two Gaussian distributions each represented by 500 samples and an ellipse depicting the 90% confidence region. The blue distribution has a much smaller covariance than the red distribution, indicating a more certain position estimate.)

Effective decision making relies heavily on covariance. The system needs to determine if its confidence in its location estimate is sufficient to proceed with its current task, or if it must take corrective action and attempt to reduce its position uncertainty. Path planners can take pose covariance as input, and this allows them to adjust movement speed as well as path safety margins.

Covariance also plays a vital role when integrating measurements from different sensors or combining pose estimates output from various internal modules, offering a systematic way to appropriately weight this information. Higher confidence data will be given more weight. This ensures that the most reliable information has the greatest influence on the system’s overall pose estimate.

It is important that the covariance that is output accurately reflects the true level of uncertainty. An overconfident could be dangerous, and a system that is too underconfident might be too inefficient.

Overconfidence

The system is overconfident if it assumes it’s location and map are more accurate than they actually are. The output pose covariance will be smaller than it ought to be, meaning the system is underestimating the probability that its actual location could be further away from where it thinks it is.

This can lead to underestimating new information. If it believes in its current pose estimate too strongly, it may undervalue new, especially conflicting, data. As a consequence it might resist adapting to new situations. This could even lead it to disregard corrective information, potentially preventing it ultimately from reducing error.

An overconfident might cause the robot to exhibit risky behaviours such as travelling too quickly, or not leaving enough obstacle clearance. This could potentially result in dangerous situations, such as collisions or the robot getting stuck.

Underconfidence

Conversely, an underconfident system will be excessively cautious regarding the quality of its pose estimate, resulting in an excessively large covariance. This means it is exaggerating the likelihood that its true location is significantly different from its estimated position.

This would likely result in reduced efficiency, or increased running times as a result from overly cautious behaviours. For example the robot might move at a ridiculously slow pace, or it might repeatedly keep deciding it requires additional data and processing time to confirm already known information.

Understanding covariance quality

It is therefore imperative that we are able to analyse and understand the quality of the covariance estimates that the system, or any of its internal modules, produces. A good covariance should accurately model the probability: the “true” pose should be contained inside the estimated covariance’s 90% confidence region 90% of the time. It is realistic to expect some degree of degradation in the covariance quality because the system is nonlinear. This means the true probability distribution, in general, can’t be perfectly modelled as a Gaussian distribution, so the Gaussian representation will necessarily be an approximation.

To perform this analysis we look at the system’s performance over a large variety of datasets, and compare it to ground-truth. Internally at Kudan we are continuing to explore better ways of measuring and visualising covariance quality, as well as trying to understand which variables have the most significant impact on covariance quality.

Once a system’s covariance quality is understood, the next step is to use this information to calibrate the uncertainty estimation: adjusting the estimated covariance in order to better represent the true uncertainty.

Closing thoughts

The management of uncertainty through covariance is fundamental to the operational success of mobile robots, ensuring both safety and efficiency in dynamic environments such as warehouses. By refining our understanding and calibration of covariance estimates, we continue pushing closer to finding the right balance between avoiding the pitfalls of dangerous overconfidence, and the inefficiencies of undue caution.

 

Please contact us for learning further technical information