The Future of Photorealistic VR Is Being Argued in a SIGGRAPH Side Room This Week. It Is Called Gaussian Splatting.

Walk the floor at SIGGRAPH this week and the loudest rooms are the obvious ones. The AI keynotes, the neural rendering demos with a crowd three deep, the headset makers hinting at what ships this fall. All of it matters. But if you want to know what virtual reality will actually look like in two or three years, skip the marquee stage and find Room 518 on Tuesday morning. That is where a group of researchers and engineers will spend ninety minutes arguing about a rendering technique with a deeply unglamorous name: Gaussian splatting.

If you have not run into the term yet, here is the short version. For decades, everything you see in a game or a VR app has been built out of polygons: millions of tiny triangles stitched into meshes, wrapped in textures, and lit by hand. It works, but it is slow to make and it always looks a little like a model of the world rather than the world itself. Gaussian splatting throws that pipeline out. Instead of triangles, a scene is stored as a cloud of soft, colored blobs. Each blob is a fuzzy three dimensional dot, a "Gaussian," with a position, a color, a size, and a transparency. Stack a few million of them together and you get something that looks less like a video game and more like a photograph you can walk around inside.

The reason it is suddenly everywhere is that you do not model those blobs by hand. You capture them. Point a phone at an object, take a few dozen photos or a short video, and software solves for the cloud of Gaussians that reproduces what the camera saw. A weekend project can turn your living room into a scene you can view in a headset. That is a very different economy from paying an artist to build the same room, one triangle at a time.

The fourth dimension is time

The session in Room 518 is not really about that part, though. It is built around the jump from 3D to 4D, and the fourth dimension is time. A static splat of your living room is a nice trick. A splat that moves, a person talking, a dancer, a chef cooking, captured as a volume you can walk around instead of a flat video, is a genuinely new medium. The agenda reads like a to-do list for making that practical: compressing these enormous point clouds, animating them, streaming volumetric media all the way from capture to a viewer's headset, and even treating splats as a long term archival format for real places. There is also a thread on using splats as a building block for world models, the AI systems that try to understand and predict physical space. That is the part that quietly connects this side room to the loud AI keynotes down the hall.

Why VR is the payoff, not an afterthought

Flat screens have always undersold Gaussian splatting. On a monitor, a splat scene looks like a very good photo. In a headset, it looks like presence. Because the representation is genuinely three dimensional, you get real parallax and real depth as you lean and move, the exact cues that make VR feel like somewhere instead of a picture of somewhere. Volumetric video is the clearest example. A concert or a family moment captured as moving splats is something you attend, not something you watch. No polygon budget, no hand modeling, just the recorded light of a real moment played back around your head.

The catch, and there is always a catch, is performance.

Drawing a few million soft, overlapping blobs at the frame rate VR demands is brutal. A standalone headset has to hit around 72 frames per second, twice, once per eye, across a wide field of view, on a mobile chip with no fan and a battery to protect. On a desktop with a strong GPU you can throw six million splats at the problem and stay smooth. On a Quest running standalone, the practical ceiling is closer to a few hundred thousand before the frame rate falls apart. Closing that gap is exactly what a lot of this year's research is chasing. Foveated approaches render splats in full detail only where your eyes are actually pointed and let the periphery go coarse, which is a natural fit for headsets with eye tracking. Others focus on smarter compression and on streaming a scene a slice at a time instead of loading all of it at once.

None of this is theoretical. You can already load moving volumetric captures on a Quest 3 or a Pico 4 today, standalone, with no PC in the loop, through tools like Gracia. Research groups have published datasets for walking through real splat scenes in VR, and companies working on world models are shipping splat based environments you explore in a headset. The pieces are on the table. SIGGRAPH is where the people building them compare notes and figure out which approaches win.

The part I actually care about

What makes this exciting to me is not the photorealism, though the photorealism is lovely. It is that the whole thing is open in a way most graphics breakthroughs are not. The core method was published in the open, the tooling is spreading through open source projects and hobbyist communities as fast as through big studios, and the barrier to capturing a scene is a phone and some patience rather than a capture stage and a budget. That is the kind of shift that decides who gets to make VR content, not just what it looks like. It rhymes with the OpenXR standards push happening a few rooms over this same week: the people who actually build this technology keep choosing open foundations even while they compete like mad on everything stacked on top.

The headset wars will get the headlines this fall, as they should. New chips, new displays, new prices. But a headset is only ever as good as the thing it is showing you. The argument happening in a side room in Los Angeles this week is about whether the next generation of VR gets built by hand, one triangle at a time, or captured from the real world one photo at a time. My money is on the photos.