(03/14/17 updated to work with Google VR SDK 1.3 and the Unity 5.6 beta) Since writing my previous Daydream tutorials many things have changed in the Daydream Unity SDK. This post is a comprehensive tutorial taking you through the steps involved in creating an interactive Daydream controller based app from scratch. By the end, we’ll have a working VR app with a controller elbow model, body model, and controller based interactivity and teleportation around a scene.
(03/14/17 update) PLEASE NOTE: I’ve updated this tutorial and the working package to use the Google VR SDK 1.3 and the Unity 5.6 beta, there are new components in this SDK that didn’t exist in previous SDKs. It’s important to update for the tut to work. Things will no doubt change with the SDK in the future so this tut may not be compatible with future SDKs, I’ll endeavor to update when I can. You can download the finished Unity package here. I was using Unity version 5.6.0b11 (5.6 beta) and Google VR SDK for Unity v1.30 see the Google VR Developer site for more info. With the 1.3 SDK you will need to use the Unity 5.6 Beta, you may encounter problems using previous versions of Unity (including the Unity Daydream Preview). Import this package into an empty Unity project and have a poke around to see what’s going on.
Get Started
The first thing to do is create a new scene in Unity, remove the MainCamera from the scene (we’ll add a camera again later) and add a ground plane to give a landscape effect. Next, add the GVRControllerMain and GVRViewerMain Daydream components to the scene. These are the two workhorse components of a Daydream app.
- GvrControllerMain: is the main entry point to the Daydream Controller API. It also allows access to tweak the Controller’s arm model depth and rotation from within the Unity Inspector.
- GvrViewerMain: Communicates with the head mounted display (HMD). Responsible for querying the device for viewing parameters, retreiving the latest head tracking data and optionally providing the rendered scene to the device for distortion correction.
Your scene’s hierarchy should look something like this:
Next, we’ll set up the Player object to house the MainCamera, controller, and anything that is physically connected to the user. Create an empty game object called Player and set its y-pos to equal 1.6 so the users head is 1.6 meters off the ground. The SDK includes a useful prefab: GvrControllerPointer (GoogleVR/prefabs/UI/GvrControllerPointer) that out of the box gives us a model of the controller, a laser pointer and fairly complex “arm model” for natural rotation.
- Arm model: An arm model (also called elbow model) is a script that moves the controller around your body naturally in the same way it would move if it was attached to your arm. Because we don’t actually know where your controller is the script guesses the position by using the tilt of your wrist to keep the controller positioned correctly.
- Body Model: Conceptually the body model keeps the controller to the side of the user, either left or right handed, and keeps it in the correct position when the user rotates their body/headset on the y-axis.
Take a look inside the GvrControllerPointer. It contains two objects: Controller and Laser. The Controller has a GvrArmModel script attached to it which super handy because it takes all the pain out of us writing out own arm model. If you keep drilling down into the child game objects of the Controller you’ll find a Tooltips object, I’ve switched this object off in the inspector because I think it’s superfluous.
Now it’s time to add a camera as a child of the Player object. In the camera’s inspector set it’s Tag to MainCamera in the drop down and update its clipping plane so that it is 0.03. If we don’t do this it will clip the controller out of view when you raise it to eye level. Your Hierarchy should look like this:
Now Build and run to your phone and test in your Daydream headset and you should see the controller move along the arm model and rotate around with your body when you rotate your head on the y-axis.
Adding Interaction
This is a good start but the app doesn’t actually do anything yet. Let’s add an object to the scene that we can interact with by clicking and hovering. I’m going to reuse as much code provided by the Daydream SDK as possible, this will include some scripts from the demo scenes, namely the Teleport script.
For the Controller to interact with objects in the scene you need to setup three things:
- An Event System.
- An Event Trigger attached to the object we’re interacting with to handle the events.
- A Raycaster to see which objects have intersected with the path of the laser.
Event System
For the Event System, we’ll use the GvrEventSystem to handle the events from the controller. This component was added to the SDK in version 1.2. Find it in GoogleVR/Prefabs/UI/GvrEventSystem and drag into the scene’s hierarchy. The GvrEventSystem contains a specialized InputModule (GvrPointerInputModule) for use with a 3D pointer from the Daydream Controller, or a gaze-based pointer for Cardboard.
Event Trigger
Add a cube to the scene and move it so that it sits somewhere in front of the camera, this is the object we’ll be interacting with. In the cube’s Inspector add the Teleport script that comes with the SDK’s demos, we’ll use this as an easy way to move the cube around randomly when clicked. Create two different colored materials and drag them into the Inactive and GazedAt fields of the Teleport script for the hover states. Now add a new Event Trigger component to the cube by selecting Add New Component in the inspector. Then select Add New Event Type in the component and select Pointer Click from the drop down. Drag the Teleport script into the event’s object field and in the drop down select Teleport.TeleportRandomly. Repeat this for PointerEnter and PointerExit events but select Teleport.SetGazedAt as the method to trigger, this will swap out the colored materials when the user hovers over the cube. The cube’s Inspector should now look like this:
Raycaster
To get the raycasting working add the GvrPointerPhysicsRaycaster script to the Player’s camera. This script provides a raycaster for use with the GvrPointerInputModule we added earlier to the EventSystem. Together they raycast and bubble up event messages on intersecting objects.
Build and Run the scene and you should see something like this:
Adding Teleportation
Up until this point we haven’t actually coded anything, it’s been the software engineering equivalent of playing with Lego blocks. However, if we want to teleport around the screen this will have to change. I’m going to update the Teleport class with a new method. We could create a new class for this code but I feel like it sits well in the Teleport class so let’s just put it there. We also need to add an EventTrigger on the ground plane to listen for the teleport click events, then pass the raycasted position of the pointer to this new method to update the player’s location. Let’s create the script first.
public void TeleportTo(BaseEventData data ) {
PointerEventData pointerData = data as PointerEventData;
Vector3 worldPos = pointerData.pointerCurrentRaycast.worldPosition;
Vector3 playerPos = new Vector3(worldPos.x, player.transform.position.y, worldPos.z);
player.transform.position = playerPos;
}
The script takes the BaseEventData object from the EventSystem, casts it as a PointerEventData type and then extracts the raycast hit position from the current raycast. We then simply assign this position to the player’s position. As well as this we’ll need to create a public reference to the player’s game object and use the Unity.EventSystems class path as an import:
using UnityEngine;
using System.Collections;
using UnityEngine.EventSystems; // ADD THIS
[RequireComponent(typeof(Collider))]
public class Teleport : MonoBehaviour, IGvrGazeResponder {
private Vector3 startingPosition;
public Material inactiveMaterial;
public Material gazedAtMaterial;
public GameObject player; // ADD THIS
Hooking it up is super simple. Add the updated Teleport script to the ground plane game object, drag the reference of the Player from the hierarchy to the player field in the Teleport script, I’ve also dragged the grounds material color to the Inactive and GazedAt fields so it doesn’t update the ground’s color. Add a new EventTrigger to the ground plane in the same way we did previously for the cube, but this time we only need a PointerClick Event, drag in the Teleport script into the EventTrigger’s object reference and choose the TeleportTo method from the drop down – it will be at the top of the list because it’s passing the BaseEventData. Your ground plane inspector should now look something like this:
That’s it! Now build and Run the scene and you should be able to teleport around as well as interact with the cube.
Hopefully, this tutorial was helpful in getting you familiar with building a Daydream app from scratch. There are different ways of doing all of the things in this tut and I’d love to hear from you in the comments if you think you have a different or better solution.
Disclaimer: I’m a Google employee and write blog posts like this with the sole purpose of encouraging and inspiring developers to start exploring Google’s amazing Daydream VR platform. Opinions expressed in this post are my own and do not reflect those of my employer. I would never share any secret or proprietary information.
Thomas
That’s super helpful to learn that you can extract the Raycast hit info from the PointerEvent and BaseEvent data! Thanks!
thepostfuturist
Thanks.
Ricky
Coming from a Software Engineering background, this is exactly what I was looking for to get me started with this kit.
Well written! (thumbsup)
@_SamKeene
Cheers Ricky!
SU
Another beginner tutorial your readers might find useful is how to tether your Pixel to the computer, and then connect the real controller for debugging in the Unity editor.
MS
Thanks! Any idea why I keep getting this log vomit?…
GvrPointerInputModule requires GvrPointerManager.Pointer to be set.
UnityEngine.Debug:LogWarning(Object)
GvrPointerInputModule:Process() (at Assets/GoogleVR/Scripts/EventSystem/GvrPointerInputModule.cs:137)
UnityEngine.EventSystems.EventSystem:Update()
I have only one project/scene working, but all other attempts result in the warning above. I’ve reread the posts multiple times and compared all settings with the one working scene, but still stuck.
Al
I ran into something similar except without the error for whatever reason :(. Maybe this will work for you: Within the GvrViewerMain add the GrvPointerManager. The latest GoogleVR SDK I downloaded does not come with the point manager component included, you’ll have to add it manually.
Drew Hunt
Thanks again, Sam….another helpful tut!! Quick question, have you experimented with trackpad locomotion yet? I know the consensus seems to be that it is too uncomfortable, but there are a few implementations of it (onward, Doom BFG) that have gone over fairly well. Curious to hear your thoughts on implementing it into Daydream
again, these are super helpful, thanks for taking the time.
Beth Brennan
This is fab – but I’m having trouble getting the raycast hit to trigger the on pointer enter event, am I missing a step between the eventsystem and the gvr code perhaps?
Harrison
I’m having the same trouble too… the click and pointer enter aren’t being triggered on the cube…
Al
I’m having the same issue, have you solved the problem or figured out the missing step?
Al
I missed to add the GVRPointerManager component to the GVRViewerMain… silly me. Thanks for this article!
Kaspar
Hey Sam. My aplauses go to you, thanks for this! However.. I keep on running into some errors. One of which is that when I build and run to my Pixel xl for the first time, my phone tells me “this cardboard app is not compatible with your daydream device”… How can that be? I then tried to just build and run your own project file, however then I get an error saying “Error building Player because scripts have compile errors in the editor”. Reading further in the console it says “Some scripts have compilation errors which may prevent obsolete API usages to get updated. Obsolete API updating will continue automatically after these errors get fixed.” I assume GVR sdk version is outdated in your tutorial project? Any guidance would be a huge pile of… karma points and etarnal gratefulness. Cause the thing with “incompatible app for daydream” keeps on popping up and I can’t seem to find a way to fix it :(.
@_SamKeene
Hey peeps thanks for the comments. There seems to be a bug for some people (but not everyone, which is strange..). Not 100% sure what this is. My wife and I are about to have a baby so we’re super busy at the moment. I’m so sorry but I won’t have much time to look into this right away at the moment. Previous problems of this nature have happened on updates to the SDK, Unity and things being moved, deprecated etc. If things have changed possibly this will require a new step in the tut.. So I’d check to see if you’re running the Unity Daydream preview GVR-13, using the 1.1 SDK and see if the imported package/scene provided works when you import into an empty project. I’ll look into this ASAP and try and replicate any problems you’re having and I apologize for the inconvenience – hope to get you up and running in VR soon!
Brent Jackson
Thanks for writing this. It’s exactly what I needed and works great.
My wife and I are also about to have a baby (May 5th) so congrats to you guys! I know how busy you are 🙂
Mike
This tutorial is fantastic — thank you so much for putting it together! I hit a snag getting events to fire, however, and found out that the regular event system did not work with the gvr pointer events. I deleted the Event System and replaced it with the GvrEventSystem that comes with the SDK and presto-changeo, events fire beautifully.
@_SamKeene
Hey Mike (and others) thanks for the comment! You are correct, This tutorial was written using SDK 1.1 but the new SDK 1.2 for Unity has a bunch of changes that will require me to update a couple of steps. In the 1.2 SDK the GvrEventSystem prefab has been added. This includes GvrPointerInputModule and GvrPointerManager in a single prefab and the GvrPointerManager has been moved from GvrViewerMain to GvrEventSystem. See the official release notes for more info: https://developers.google.com/vr/unity/release-notes. I’ll update the tutorial as soon as I have time, in the meantime it seems the fixes are relatively simple, follow steps outlined in the other comments. OR just use the supplied package containing the old SDK and import into an empty Unity Daydream Preview project and that should work.
collaby
How do you get the recticle to grow in size when it hovers over an object?
collaby
I mean reticle*
@_SamKeene
Hey peeps, I’ve updated this tut to work with Unity 5.6 beta and the Google VR Unity SDK 1.3. The main change is that it is now using the GvrEventSystem component.
SNS
Thanks! This is what I’ve been looking for.
Luis
Hi! Thanks for this tutorial!
One question: how do I change the controller 3D Model? For example, I want to be holding a sword instead of the controller.
@_SamKeene
Hi Luis, thanks for the comment. You can easily replace the controller model by switching off the ddcontroller in the Controller’s hierarchy and adding your own gameobject.
Luis
Thanks!! Sorry for late answer. Again, thanks for the tutorials!
Luis
Hey Sam, I got another question (sorry for noobness).. I can’t click a button with the controller LaserPointer, the button highlights and becomes clickable only if I’m looking at it (if the reticle is over it). How do I change that? How can I make it work just with the controller and not with the gaze?
Netves
Hey Luis, you probably have gotten it fixed by now, as I was the exact problem and found out hours later that you need to switch off the GazeReticle (the Looking Reticle) for the LaserPointer to work. I will leave it here for others
Michael Martin
Hi Sam,
I’ve been playing with your example and think its fantastic. I’m building a little environment that the viewer can walk around with areas that the player can walk and cannot. It all works well but I’d love to be able to add a ground pointer, so that when you point to an area of the ground it can show if you are able to move there or not. Would you be able to give me any tips on how best to achieve this? Best regards
Andrew
Hi Sam, I recently stumbled upon your extremely useful site in search of a solution for the issue I have. Your basics have been very helpful! Seeing that you have some experience in Daydream I was wondering if you could nudge me in the right direction. Here’s my question on the Daydream Unity forum https://forum.unity.com/threads/transforming-daydream-controller-input-into-a-normalised-2d-space.513100/#post-3359712
Essentially, is it at all possible to transform input from a daydream controller into a normalised 2D space? I understand that you value your time as a developer so I’ll gladly pay for the advice if you can provide any. Just to clarify, this is by no means a commercial project and I’m just a student trying to figure out GVR.
Mike Stein
Any chance you’ve got any advice on getting both a controller *and* a gaze pointer working in the same scene?
Jordan Neel
This was extremely helpful. Thanks so much!