Do inside-out headsets need base stations?

No. Steam Frame, for example, does not require or support SteamVR base stations and tracks itself and its controllers with four onboard grayscale cameras. Base stations belong to outside-in systems such as the Valve Index.

Why does inside-out tracking struggle in the dark?

Camera-based tracking needs visible detail. Some headsets add infrared illuminators so their cameras can see the room without visible light, which is why Steam Frame can track in a dark room. Others rely on visible light and degrade when the scene gets too dark or too uniform.

Is inside-out tracking as accurate as base station tracking?

For head pose inside the tracked volume, modern standalone tracking is stable enough that reviewers describe it as essentially perfect. The practical differences show up in controller tracking volume, occlusion resistance and behaviour in feature-poor or reflective rooms.

Which headsets use inside-out tracking?

Meta Quest 3, Apple Vision Pro and Valve Steam Frame all track inside-out. Vision Pro uses six world-facing tracking cameras plus a LiDAR scanner, while Steam Frame uses four fisheye cameras with infrared illuminators.

Does inside-out tracking use the same cameras as passthrough?

Often yes, but not always the same sensors. Steam Frame reuses two of its four tracking cameras for monochrome passthrough, while headsets with color mixed reality add separate RGB cameras for the view you see.

What Is Inside-Out Tracking in VR?

Q: What is inside-out tracking in VR?

Inside-out tracking is a method where cameras and motion sensors mounted on the headset observe the surrounding room and compute the headset's position from what they see. No external base stations or markers are required.

What is inside-out tracking?

Inside-out tracking is a way for a headset to work out where it is by looking outward at the room. Cameras and motion sensors are mounted on the device itself, and software compares what the cameras see from one moment to the next to estimate how the headset moved. Nothing in the room needs to emit a signal, and nothing needs to be installed on the walls.

The alternative is outside-in tracking, where emitters placed in the room track sensors on the headset. Valve’s Index is the clearest recent example: UploadVR describes its tracking as precise laser tracking with wall-mounted base stations, sold as part of a premium tethered kit UploadVR: Steam Frame isn’t Valve Index 2. The Index has since been retired in favour of a standalone design, a shift covered in Valve retires the Index.

The trade is straightforward. Outside-in buys accuracy and a large tracked volume at the cost of setup, cabling and a room permanently instrumented with hardware. Inside-out buys portability and zero setup at the cost of putting the sensing problem, and its compute bill, on the headset.

Inside-out vs outside-in

Outside-in: base stations or cameras in the room observe the headset and controllers. Tracking quality is high and predictable, but you need line of sight to the emitters and a fixed play space.
Inside-out: the headset observes the room. Setup is a boundary drawing, and the same headset works in a new room in under a minute.
Hybrid: a headset can track itself inside-out while accessories use their own sensors, which is common for trackers and some controller setups.

How inside-out tracking works

Cameras observe the room

The cameras are usually wide-angle fisheye lenses aimed outward, because a headset needs to see enough of the room to survive fast head turns. Steam Frame uses four outward-facing grayscale fisheye cameras for headset and controller tracking, two on the top corners and two near the bottom front, and explicitly performs SLAM with computer vision rather than base stations UploadVR: Steam Frame features and details. Grayscale is a deliberate choice: it carries the contrast that feature detection needs without the bandwidth and compute of full color.

Those cameras are not only for tracking. On Steam Frame, two of the four double as monochrome passthrough, and headsets with color mixed reality add dedicated RGB sensors. The overlap and the differences are worth understanding if you build mixed reality content, and how passthrough works covers that pipeline in detail.

The IMU covers the gaps between frames

Cameras run at tens of frames per second. Head motion happens faster than that, and a frame or two of latency is visible as swimming. So every headset pairs the cameras with an inertial measurement unit: a gyroscope for angular velocity and an accelerometer for linear acceleration, sampled hundreds or thousands of times per second.

The IMU alone cannot track position, because integrating acceleration accumulates error quickly. The cameras alone are jittery and drop out when the view is blurred or blocked. Fused together, the IMU carries the pose through the gaps between camera frames while the cameras correct the drift. This combination is what the literature calls visual-inertial odometry.

SLAM builds a map and localizes against it at the same time

Simultaneous localization and mapping is the part that makes tracking persistent. The system extracts distinctive features from each camera frame, matches them against previous frames and a sparse map of the room, and triangulates where those features sit in 3D. As the user walks around, new features are added and old observations are refined.

The open-source state of the art shows how far this has come. ORB-SLAM3 handles monocular, stereo and RGB-D cameras with pin-hole and fisheye lens models, runs a tightly integrated visual-inertial system, and reports an average accuracy of 9 mm under quick hand-held motion in a room-scale dataset the authors describe as representative of AR/VR scenarios arXiv: ORB-SLAM3. It also survives losing tracking by starting a new map and merging it later when the user revisits a known area, which is the behaviour you feel as fast relocalization after the headset is taken off and put back on.

Prediction and reprojection hide the remaining latency

Even a fast pipeline has latency: capture, feature matching, fusion, simulation, rendering, scanout. The runtime therefore does not ask the application for the current pose. It asks for the pose at the moment the frame will actually be displayed.

In OpenXR that value is predictedDisplayTime, and the specification is blunt about why it matters: an accurate predicted display time is important to avoid black pull-in by reprojection and to reduce motion judder, and “the better the predicted display time, the less latency experienced by the user” Khronos: OpenXR specification. The compositor then takes the rendered frame and warps it to the latest pose at scanout. Prediction plus reprojection is why a standalone headset with a mobile chipset can still feel solid, and why a rendering engineer’s frame budget is really a latency budget.

How controllers get tracked

Almost all inside-out systems track controllers optically. The controllers carry infrared LEDs, and the headset’s cameras solve their pose from the pattern. Meta’s Touch Plus controllers moved those LEDs from a tracking ring onto the controller body, and UploadVR found the result essentially identical to Quest 2, with a wider horizontal range and a narrower vertical range because of where the headset cameras sit UploadVR: Quest 3 review.

LED count is a design lever against occlusion. Touch Plus controllers have eight IR LEDs each, while the Steam Frame Controllers have eighteen spread across the face, handle and bottom, which Valve’s design should make more resistant to being blocked by a hand or a body UploadVR: Steam Frame features and details. Between camera frames, controller pose is predicted with the controller’s own IMU, which is why tracking can briefly survive going out of view and why it drifts when it stays there.

Hand tracking drops the controller entirely and runs computer vision on the same cameras. That is a different problem with a different failure mode, and the camera placement that suits controller LEDs is not always the placement that suits hands. Steam Frame does not currently support controller-free hand tracking at all. Eye tracking is a separate pipeline again, using cameras aimed at the eyes rather than the room; how VR eye tracking works explains that side of the sensor stack.

Reference spaces: how applications see tracking

An application never touches raw tracking data. It asks the runtime for poses in a reference space, and OpenXR defines the ones that matter Khronos: OpenXR specification:

VIEW tracks the view origin itself and is not gravity aligned. It is for head-locked content and targeting rays, not for rendering the world.
LOCAL is a world-locked, gravity-aligned origin that excludes pitch and roll. The runtime may pick the launch position or a calibrated zero point, and may silently recenter it.
LOCAL_FLOOR is the same idea with the origin placed at an estimate of floor level, and runtimes must support it.
STAGE is a runtime-defined flat rectangle you can walk around, with the origin on the floor at its centre. It is optional, and it only exists once the user has defined a play area.

Recentering is the practical consequence. When a user holds the recenter command, the runtime moves the LOCAL origin and every application inherits the change without special handling. If your content assumes the origin is fixed in the room, recentering will break that assumption. Android XR headsets inherit the same model through OpenXR, which what Android XR is covers from the platform side.

What shipping hardware looks like

Meta Quest 3 tracks with outward-facing cameras plus a depth projector that also powers room scanning and boundary suggestions. UploadVR found head and hand tracking working in almost total darkness, a clear improvement over earlier standalone headsets.
Apple Vision Pro lists six world-facing tracking cameras alongside a LiDAR scanner, TrueDepth camera, flicker sensor and ambient light sensor, plus four eye-tracking cameras Apple: Vision Pro specifications.
Valve Steam Frame uses four grayscale fisheye cameras with infrared illuminators so tracking survives a dark room, no base stations, and two front cameras for monochrome passthrough.

How those choices land in practice is the subject of Quest 3 vs Steam Frame, which compares the two tracking and streaming designs side by side.

Why it matters for graphics and XR work

Tracking is not free. Feature extraction, matching and fusion run on the same chipset, memory bus and thermal budget as rendering, and they run every frame whether the scene is simple or complex. That competition is one reason standalone headsets lean on foveated rendering, dynamic resolution and reprojection rather than brute force.

It also changes what “correct” means in a renderer. Camera poses arrive with timestamps, not as instantaneous truth, so anything that depends on precise alignment with the real world, such as anchoring a virtual object to a table, depends on the runtime’s map quality as much as on your content. Depth sensors and scene meshes help, but their resolution is coarse: UploadVR measured Quest 3’s room mesh cells at roughly 5 to 10 cm, which is enough for boundaries and not enough for pixel-accurate contact.

Finally, tracking quality is a product decision. A headset that tracks in a dim room, or in a room with a glass table, is usable in more places than one that does not. That is why illuminators, camera placement and LED counts keep showing up in hardware specifications that most users never read.

Limitations and caveats

Light and texture. Cameras need contrast. Blank walls, uniform carpets and very dim rooms starve the feature detector, and infrared illuminators only partly compensate.
Reflective and transparent surfaces. Mirrors, windows and glass tables confuse depth and mapping. UploadVR’s review notes the glass table in their living room would not scan and windows could throw the room setup off.
Occlusion. Controllers behind your back or under a blanket lose optical tracking, and the IMU can only carry the pose for a short time.
Tracking volume shape. Camera placement creates a volume that is not a sphere. Quest 3’s controller tracking is wider to the sides and narrower overhead than Quest 2’s.
Drift and relocalization. After tracking loss, the system has to recognise where it is. Fast recovery depends on having seen the area before.
Power and heat. Every camera and every millisecond of vision processing is budget taken from rendering and battery life.

Sources

Khronos: OpenXR specification 1.1 (reference spaces, predicted display time, reprojection)
Apple: Apple Vision Pro specifications (world-facing tracking cameras, LiDAR, TrueDepth, eye-tracking cameras)
UploadVR: Valve Steam Frame official announcement features and details (four fisheye cameras, IR illuminators, no base stations, controller LED counts)
UploadVR: Meta Quest 3 review (Touch Plus IR LEDs, tracking volume, depth projector, low-light hand tracking)
UploadVR: Steam Frame isn’t Valve Index 2, and that’s a good thing (Index laser tracking and base stations)
arXiv: ORB-SLAM3 (visual-inertial SLAM, fisheye models, accuracy in AR/VR-like conditions)