Prototyping Xbox and Oculus VR

In mid 2015 the TV and Xbox Incubation group started considering if Xbox OS could be adapted to a VR space. How would the controller work in a VR space? Could you do multiple things at once, like play a game and talk with your friends? How would you access all the content you already have on your Xbox. These questions kicked off a 4 week prototype excersise with storyboards, images, and animations. 

Translating a 2d interface into 3d space

Xbox is organized in a classic pivot structure and a content selector operated by the controller. Xbox has employed this pivot and grid model since Xbox 360, and our team decided to mimic the same structure within VR, regardless of the final UI model.

Anchoring the user in space

Translating Xbox’s 2d content into a 3d world proved simple enough. The larger problem in user tests proved to be defining scale, familiarity, and context in a potentially limitless space. Key idea involved exploring:

• A living room with content arranged among furniture
• A theater with content surrounding you
• A floating platform

initial storyboard concepts mocked out content and space ideas

Early investigations started with simple 3d models generated in SketchUp and used for mocking up content experiences. The images were used to simulate the experience of a VR interface and make test some basic assumptions about field of view, content selection, familarity and navigation.

Doing two things at once in VR

Parallel tasks, like watching a movie and reading a message is one of the core challenges and opportunities with immersive VR. How could other aspects of the Xbox ecosystem like friends and party invites appear in a 3d environment. How could simple things notifications come to life?

The team identified Xbox messaging (chatting with friends, party invites) as a good litmus test for gameplay.

early story boards showed a series of messages brought into focus while the main content remained in the background
An early example of a message thread coming into focus while playing a game.
given the the amount of space in VR, messages could feature animated avatars

Receiving messages and inviting friends

Peripheral space could be used in VR to talk with friends via their avatars. Here a game is paused and the user switches context to talk with a friend.
In-game invitations shown here to function via a heads-up-display to minimize context diruptions
Other parallel experiences beyond messaging and party invites involved augmenting game experiences, like an interactive map during a driving game.
Achivements and other important information could be within your field of view.

Synthesizing the concept

The final design focused on three key problems identified throughtout the prototyping: selecting content, scrolling content, and changing focus. The model closely matched the Xbox OS pivot structure with panels fixed in an open space. This allowed a user to maintain a reliable cursor point in the open space while looking around at adjacent content or events like messaging. 

example of scrolling and focusing on a piece of content within VR space
example of changing focus and bringing a message panel forward


After several tests within Unity 3D, one of the repeating feedbacks was the desire to put down the controller and touch things. Although we had no hardware support at the the time of the prototype, this led us to a final exploration of a touchable ‘tablet’ interface that could support a future hand-based UI.