Mechanism of SLAM (2/3)

Previous Article:
Mechanism of SLAM (1/3)

Tracking

The tracking thread provides real-time localisation by extracting the relevant information from the main camera image and from the auxiliary information, if available, and subsequently compares the extracted information with the existing map structure.

In each frame, the image is analysed to extract the feature points that are the distinctive features visible in the image. To do so, the corner shapes are detected at various scale levels in the image, and each of them is further described based on an observation of the feature point.

While initiating a SLAM system, the map of the environment is empty, and its first keyframe should be created. This can be performed in different ways depending on the available sensors.

If the map is not empty, the tracker can use the most recent keyframes as references to compute the new camera position. The observed feature points are compared with the observations in the reference keyframe, and the matching pairs are used to retrieve the relative motion of the camera in space.

The tracking thread may decide to ask the mapping thread to expand the map using the latest position based on the amount of movement of the camera.

It may also happen that the camera gets shut or rotated too suddenly for the new frame to sufficiently overlap with the existing map keyframes. In that case, tracking is lost, and there are two possible options depending on the SLAM use case: the map could be cleared and a new SLAM sequence begins or, more commonly, the system can attempt to relocalise the camera against the map.

During relocalisation, the new camera frames are visually compared with a database of the full map in search of a keyframe that is sufficiently similar to act as a candidate for resuming camera tracking. Various kinds of visual metrics can be used for comparing the image similarity in either the local or global case.

If a candidate keyframe is obtained for relocalisation, tracking is tentatively resumed from its pose; for the following few incoming frames, the tracking quality is thoroughly checked to ensure that the candidate was the accurate one. If these checks are successful, SLAM continues normal tracking and mapping; otherwise, it keeps trying to relocalise against all the map. During the trial period, it is never required to expand the map to ensure that it is not potentially corrupted.

Next: Mechanism of SLAM (3/3)