Cameras and sensors

HoloLens contains an array of sensors and cameras that accumulate data from the real environment. In the top-front part of the HoloLens, there are several front-facing cameras and sensors that are used to scan the environment and collect as much data as possible.

Sensor Bar placement in HoloLens Device

Sensors pass the data to processing units, which make it understandable by the HoloLens. At a high level, the sensor bar on the HoloLens embraces the following:

Depth Camera
Ambient Light Sensors
Environment Understanding Camera
Video Camera

The following figure illustrates the different types of sensors placed in the sensor bar:

The Sensor Bar – Sensors and Camera

It all starts with the Depth Camera. The HoloLens Depth Camera is based on the Kinect technology, which uses a combination of camera and infrared depth-sensing vision to draw the digitalized surroundings by measuring the distances and depth.

Kinect has been a game changer in the world of motion games and applications. It is a motion sensing device, which was originally developed for the Xbox 360 gaming console. Kinect provides a Natural User Interface ( NUI) for interaction using body motion and gestures as well as voice commands. It is a horizontal device with depth sensors, color camera, infrared ( IR) Emitter, and a set of microphones, all secured inside a small box. This small box is attached to a small motor and tilts the device in a horizontal direction. When it is working, the IR Emitter constantly emits infrared light in a pseudo-random dot patterns over everything in front of the device. IR dots are invisible to us, but IR depth sensors can read those dots. The doted lights reflect from different object, and the IR depth sensor reads from the objects and converts them into depth information by measuring the distance between sensor and the object from where the IR dot was read.

HoloLens device has a wider field of view and looks out at the world with a field of sensing 120x120 degrees.

View area for HoloLens

Ambient Light Sensors maximize visual quality and energy conservation within the device, depending on the change in environment lights. An ambient light sensor is mounted in front of the HoloLens to detect the environment's light quality and intensity, which help the device in controlling the brightness. This helps to reduce power consumption, extend battery life, and provide optimum viewing in diverse lighting conditions.

The Environment Understanding Camera builds a quick understanding of the environment and provide the basis for head tracking and surface reconstruction. This is the key to understand your physical world and place holograms on physical objects.

HoloLens has a 2MP HD Video Camera that is used to record and share the visuals captured by the device and capture real-time photos. Camera feed can also be used to share the visual experience among different users.