What is Gaussian Splatting?

We’re constantly bombarded with new tools and technologies that push us closer to a future where physical stores, online experiences, and digital entertainment seamlessly blend. This futuristic vision is rapidly becoming reality thanks to advancements in computer graphics. One technique generating a lot of buzz lately is Gaussian Splatting. But what exactly is it, and what does it do? Let’s break it down

A New Way to Capture 3D Scenes

Some of us might be familiar with 3D photogrammetry, the technology used to create realistic 3D models from photographs. You may even have tried photogrammetry apps on your phone where you can “stitch” together a number of photos or a video into a textured 3D model. It’s a powerful tool, but it can be computationally expensive to run. 

Gaussian Splatting takes a different approach. Instead of building a 3D mesh, it generates a point cloud using millions of tiny 3D Gaussians – bell-shaped curves in 3D space. Each Gaussian has properties like location, size, and color, essentially acting like a tiny paint splat contributing to the overall image.

NeRF vs. Gaussian Splatting

Both NeRFs and Splats are cutting-edge technologies that can be used within a neural rendering workflow. However, they take different approaches. Neural Radiance Fields (NeRF) typically use neural networks to analyze data and learn the underlying structure of a scene. This learned information is then used to generate new views. Maybe you’ve seen examples where you’re in a 3D view and can “see behind” objects that were not initially a part of what the camera captured. Gaussian Splatting, on the other hand does not rely on AI. It represents a scene using millions of 3D Gaussians. While it might be used in conjunction with neural networks for tasks like capturing initial data or refining the final output, Gaussian Splatting itself focuses on efficiently rendering the scene with these 3D Gaussians.

Gaussian Splatting is generally considered faster and more efficient than NeRFs, making it ideal for real-time applications e.g. VR. Gaussian Splatting can handle complex lighting effects more effectively. However, NeRF might generate higher level detail.

Can I Make Gaussian Splats Myself?

Yes! There are tools that allow techies as well as non-techies to experiment. Here are a couple of examples:

• Luma AI: This platform allows you to capture 3D objects using your smartphone and convert them into splats.
• Polycam: This app lets you capture gaussian splats using your phone.

Where can I Import splats?

You can play with importing your gaussian splats in software like Spline, Blender, Unity or Unreal.

How Gaussian Splatting and Similar Tech Can Change Retail 

In a post pandemic paradigm we have seen the importance of reaching people everywhere, even in their homes. And brick-and-mortar is definitely transforming – some claim that stores in the future will likely work as showrooms rather than a place where you go and try something on. Brands and retailers all over the world are looking for ways to engage their customers by building out virtual branded experiences e.g virtual try-ons and virtual showrooms with high level of realism.

Each implementation involving 3D and interactivity have different tech requirements. In real time applications where traditional modelling becomes time consuming but level of realism is high, gaussian splatting is a promising technique providing photorealistic intricate details and textures.