Facebook’s double ambitions in virtual and augmented reality could be meeting in the middle with the Oculus Quest 2. A new developer toolkit for the Quest allows blending of passthrough cameras with VR graphics, enabling mixed reality.
Mixed reality has been bubbling up on the Oculus Quest for a while: In addition to the headset’s way of scanning the world and painting play boundaries for VR using the black-and-white cameras on the headset, Facebook has added a few modes where web browsing and typing on a Logitech keyboard work with passthrough camera data, too. Also, there’s hand-tracking.
Color effects can be applied to the black and white camera data, too.
The developer mode, called Passthrough API Experimental, isn’t intended for actual apps yet, but Facebook’s aiming for Oculus Quest apps to add mixed reality later this year. The suggested uses are for work, social and gaming. Projecting avatars onto the real world is reminiscent of what Microsoft Mesh is promising using HoloLens 2. But with the Oculus Quest 2, it would be done on a $300 consumer headset. Another area this could make a ton of sense would be fitness games, where motion-intensive games like Beat Saber could be played while layered over a passthrough view of the actual room — without the risk of accidental wall-punching.
Facebook’s post stresses that apps using the passthrough camera data can’t access, view or store images or videos using the Quest 2 cameras, an element of data privacy that the company also stressed when announcing its push into ads in VR.
The Quest 2’s cameras, however, are low-res and black and white, not ideal for capturing much real-world data or detail. This suggests a Quest upgrade in the future with better cameras: Mark Zuckerberg has confirmed a Quest Pro is in the works, which could end up leaning further into mixed reality. And with Facebook’s AR glasses plans still far from being realized, the Quest looks to be step toward figuring AR out more with current apps.
