Sensor-Independent Autonomous Mobility

Sensor characteristics like the resolution and field of view affect the resulting pixel-by-pixel representation of a given object like a car or a pedestrian. Thus, a neural network trained to distinguish these features in the images from a specific camera will often not perform as well on images from a different one. When implementing a new sensor, training must therefore start from scratch with images from that sensor, which in turn also implies the costly real-world recording and annotation of these images.

The invention defines a deflection metric whereby every pixel of an image is assigned an angle with respect to the axis of projection (see image on the left). Geometric properties of the sensor such as its field of view and nonlinear distortion are thereby encoded into the image in the form of an additional channel and taken into account by a neural network trained on images from sensors with differing properties. This in turn improves performance on images from any sensors with geometric properties within the ranges of the training set. Moreover, ambiguity in scale and distortion of objects recorded by any given sensor are also resolved.