XR's Next Platform War Is Over the 3D Compositor

The API got standardized. The hard part did not.

On October 19, Khronos published two ratified extensions, XR_EXT_spatial_container and XR_EXT_spatial_container_self_rendering, closing 18 months of work between runtime and engine vendors. The press read was that OpenXR finally has windows. That is true and it is also the least interesting part.

What the working group actually did was agree on the shape of the contract between an app and a system that both want to draw in the same room. A spatial container has a bounded mode for multitasking and an immersive mode that takes the full view. Its position is controlled by the runtime, not the app. It can be shown, hidden, and selected for input the way a window can on a desktop. The self rendering extension lets an app draw into that container using OpenXR composition layers.

Notice what is missing. There is nothing in either extension about how many containers a device can afford, at what resolution each one is drawn, how their lighting is reconciled, or who owns the pose when the budget runs out. Those questions are the platform. They are now the only part of the problem that a vendor can still win or lose, and every major XR company is answering them in a different way this month.

Three architectures, shipped weeks apart, that are not the same thing

Meta’s answer is to wrap the existing 2D Android shell and pull it outward. The Meta VR Layout SDK puts SpatialScene and SpatialWindow into a normal Jetpack Compose or React Native app, so an app that has never drawn anything spatial gets satellite panels around its main window while staying non-immersive and keeping its existing activity model. Meta is explicit about the ceiling: the Layout SDK cannot spawn 3D objects or control the environment, and volumetric content still requires the Spatial SDK. It ships with the Gaze SDK for look and pinch disambiguation and a Compose design system, the VR UI Set SDK, which Meta says is optimized to be implemented by an AI coding agent.

This is the cheap path and it is deliberately cheap. It moves millions of existing phone and tablet apps one step toward spatial without asking Android developers to learn a graphics runtime. The daily coverage of those SDKs has the API detail.

Pico’s answer is to rebuild the renderer into the operating system. Pico OS 6 is built around the Pico Spatial Engine, which the company describes as shifting “the rendering pipeline from the app layer directly into the system core,” taking “unified control over the lighting and textures of every single pixel” while “dynamically allocating CPU and GPU resources to let multiple 3D and 2D apps run in harmony.” That is not a window manager bolted onto an app model. It is a claim about owning the compositing and lighting of the whole frame, including passthrough, and it comes with the silicon to back the claim: a custom XR chip with a claimed 12 millisecond end to end mixed reality latency, working alongside a flagship SoC that Pico says more than doubles XR2 Gen 2 CPU and GPU performance, driving dual 4K micro-OLED panels at roughly 40 pixels per degree average and up to 45 in the center.

Khronos, meanwhile, wrote down the contract and left the implementation to the vendors. Bounded or immersive, runtime placed, system arbitrated input, and self rendering through composition layers. The working group also says it is building extensions so an app can hand its rendering to “a system provided renderer that is optimized for the particular hardware,” from high performance headsets down to low power devices. That sentence is the most consequential one in the announcement, and I will come back to it.

Apple is the fourth case, and the awkward one. visionOS has had this shape since 2024, with a Shared Space that holds windowed and volumetric apps under system control and a Full Space for immersion. Apple also still does not implement OpenXR, so the standard is codifying a model Apple shipped first, and Apple gets no portability benefit from it.

What a bounded 3D window actually costs

The reason the compositor is where the fight moved is arithmetic. In the immersive model, one app owns the frame. It sizes its swapchains to the runtime’s recommended resolution, chooses its foveation pattern, and submits one or two projection layers. The system compositor is a thin late stage warp plus a passthrough blend. In the container model, the system asks each container for a recommended resolution per frame and composes several of them at once.

That changes four budgets at the same time.

Memory and bandwidth first. Every additional 3D container that renders its own scene needs its own color and depth targets resident, and they cannot be recycled while the user can still see them. This is the same constraint that already forced XR headsets to render well below native panel resolution, just multiplied by the number of live surfaces. The numbers behind that squeeze are not gentle: the panels on a 4K per eye headset are far beyond what the memory system can feed at frame rate, so the OS has to divide a fixed pool between windows instead of giving it all to one.

Fill rate second, and this is the part that hurts. Foveated rendering works because a single app owns the whole view and can spend samples where the fovea is. With four windows scattered across the field of view, the eye is only ever looking at one of them, but the OS cannot retroactively un-foveate a container that was rendered while the user’s gaze was elsewhere. Either every window gets rendered at a quality level suitable for being looked at directly, or the compositor needs gaze and scene knowledge to prioritize. That is exactly why gaze tracking and window arbitration are becoming the same problem, and why eye tracking is turning into an input and a scheduler at once.

Latency third. An immersive app can be late latched: the runtime reprojects its last frame against a fresh pose the app never sees. A bounded container is not allowed to own that path, because the system has to composite several apps on one timeline. The OS owns pose to photon for everybody, which is good for consistency and bad for any developer who tuned their frame pacing assuming control of the reprojection stage.

Correct blending fourth. Pico’s phrase about taking control of “the lighting and textures of every single pixel” is a description of this problem, not marketing. If three windows and a passthrough layer share a frame, something has to decide how virtual surfaces pick up the room’s lighting, in what order alpha is resolved, and how depth interacts with the real world. Done per app, you get three different lighting models in one view. Done in the compositor, you get coherence and you lose per app authorship.

Meta’s own documentation is the clearest evidence that this budget is real and tight. Its best practices page tells developers to design for “one main window and no more than two simultaneously important spatial windows,” and to “treat capacity as platform-owned and variable.” The React Native integration reserves two spatial window slots. The Spatial Simulator allows three. The same page forbids transparency outright: spatial windows must be filled with an opaque background. It also warns that every geometry change on a window crosses a process boundary into platform window management, so animating a window’s size frame by frame is off the table.

Read that list again. On the platform with the largest XR install base, a spatial window in October 2026 is opaque, comes in quantities of two, and costs a process round trip to resize. That is not a software limitation anyone chose for aesthetic reasons. It is what the compositor can currently pay for.

The standard commoditizes the call, not the budget

This is where the OpenXR news changes competitive strategy rather than developer convenience.

What it commoditizes is the code path. Godot says it “will support spatial containers in its upcoming release.” Unity says it wants build once, reach everywhere. Varjo and Collabora are on the record supporting it. Google’s XR platform and SDK lead Jared Finder states that Google is “excited to bring Spatial Container support to Android XR.” A studio can now target one windowing model across runtimes instead of writing three.

What it does not commoditize is what happens when capacity runs out. Meta’s answer to that is a priority integer that apps assign to their own windows, with lower priority windows displaced to their fallback behavior when all slots are full and returning when one frees up. That mechanism is a vendor API. Nothing in the Khronos extensions standardizes arbitration between apps, because arbitration is a policy, and policy is where platforms differ.

So the lock-in has moved down a layer, into the same place driver specific behavior lived during the OpenGL era. Two devices can both claim spatial container support and deliver wildly different experiences, one running three lit volumetric windows over passthrough at 90Hz and the other running two opaque rectangles. The compatibility checkbox will be meaningless on its own, and developers who plan capacity from the spec sheet will be surprised.

There is also a quiet irony in the order of events. Meta’s quote in the Khronos announcement comes from Nikita Lutsenko, a distinguished software engineer in Wearables, and it talks about computing “present throughout the day, where multiple applications coexist naturally in the space around us.” That is the same company whose documentation caps an app at two spatial slots while it prepares a $1,300 pair of VR glasses that ships without a controller and runs on a tethered puck. The glasses are a bounded mode device by design: small field of view, gaze and pinch input, no room scale. The two slot limit is not a temporary embarrassment. It is a preview of the product.

The catch: the fallback renderer is the real tell

The strongest objection to this whole thesis is that standardized containers might just produce standardized mediocrity, and Khronos has already signaled how.

XR_EXT_spatial_container_self_rendering covers apps that draw their own content with composition layers. That is the path for engines and for anyone with a look worth protecting. The extension that does not exist yet is the system provided renderer, described in the announcement as optimized for hardware ranging from high performance headsets to lower power devices. In practice that is a renderer that draws your scene for you, with your app handing over geometry and state.

For compatibility, that is excellent. It is how a lightweight pair of glasses runs a hundred apps without a hundred engines competing for the same 15 watts. For fidelity, it is a ceiling. Any effect that depends on your own pipeline, custom shading, unusual reprojection, a specific depth prepass, is at the mercy of a system renderer’s feature set. The industry has been here before with browser compositing and with mobile GPU drivers, and the outcome was the same: portability at the low end, differentiation only for those who ship native paths.

There is a design risk too. Opaque rectangles placed in depth are not volumetric content, they are monitors with better ergonomics. If the dominant spatial app model becomes a Compose panel floating at 1.2 meters with two satellite cards, mixed reality becomes a very expensive monitor farm, and the volumetric case that makes XR worth wearing gets pushed back into the immersive mode where it started. Meta has an obvious route out of this: the spatial container model is close to a second attempt at Augments, the system managed 3D app category it announced in 2023 and then rebuilt from scratch after Andrew Bosworth said the original approach “wasn’t good enough.” UploadVR makes exactly this connection, and it is plausible, but Meta has not committed to supporting the extensions at all in its statement.

What to build now

For anyone shipping into this transition, the practical conclusions are unglamorous and specific.

Treat the immersive mode as the primary experience and the bounded mode as a second surface, because that is the order in which platforms will get good at them. Plan your layout for two windows, not five, and degrade to a single window gracefully. Keep window geometry static and animate the content inside it, since Meta’s own guidance says each geometry change crosses a process boundary. Assume you do not own pose to photon, and budget your frame time for a compositor that may be reprojecting three other apps. And design opaque until transparency exists, because on the biggest platform it currently does not.

If you want the portability, the Khronos sample is published and the registry entries are live, and the Android XR platform model is the closest thing to a neutral description of where the app model is heading across vendors. Pico’s WebSpatial SDK is worth a look for the adjacent reason: it is an open source attempt to make a website run as a spatial app across Pico OS, visionOS, and Android XR, which is the same bet on portability from the web side.

The benchmark for 2027

The next round of hardware will be judged on a specification nobody prints on the box: how many independently rendered 3D surfaces the compositor can hold, at what resolution, over passthrough, without dropping the pose.

Meta VR Glasses arrive in spring 2027 at $1,300 with 2.4K micro-OLED panels, around 100 grams, and a Snapdragon Reality Elite puck. Project Swan arrives with dual 4K panels and a renderer that was moved into the OS on purpose. Both are chasing the same demo, which is a browser, a video call, a 3D model and a game sharing one room. Whoever makes that feel like a workspace instead of a slideshow wins the generation, and it will not be decided by field of view.

Khronos just made sure the demo can be written once. It cannot make the demo run.

---

Sources

• Khronos: OpenXR Spatial Containers Bring a Cross-Platform Spatial Windowing API to XR
• UploadVR: OpenXR Spatial Container Extensions Standardize Volumetric App Multitasking
• UploadVR: Meta Expands SDKs To Boost Spatial Apps On Quest & VR Glasses, Including NBA, Disney+
• Meta Developers: Meta VR Layout SDK overview
• Meta Developers: Meta VR Layout SDK best practices
• Meta Developers: Window priority
• PICO Developer: PICO OS 6, PICO Spatial Engine and Project Swan
• UploadVR: Meta VR Glasses Officially Announced, Shipping Spring 2027 For $1300
• Godot Engine: Godot XR Community Game Jam VI