Sharing Augmented Reality Experiences Using Google Cloud Anchors in ARKit for iOS
Google I/O was few weeks ago where Google announced a lot of amazing technologies. As an AR enthusiast, I was really excited by the annoucement of cloud anchors. Cloud anchors will let users share their augmented reality experience with other users. Cloud anchors is part of the ARCore framework but it is available for Android and iOS. In this post I will walk you through a prototype I build which showcases the power of cloud anchors.
NOTE: There is a lot of code involved in this project. Not all code is part of this post. I highly encourage you to download the code here or at the end of the post. You will need to insert your own API key for Firebase project as well as for the Google Cloud Anchors.
What we will be building
Instead of talking about it let me just show you the demo I created. Check out the video below:
The phone on the left is controlled by me and the phone on the right is controlled by my daughter. We are looking at the same exact cube, through different phones and interacting with the same cube by changing the color of the cube. If I change the color, she sees it and if she change the color I see it.
Create a new Xcode Augmented Reality project and setup the Podfile to download the required CocoaPods. Copy and paste the following pods in your Podfile and then run “pod install”.
pod ‘Firebase/Core’pod ‘ARCore’, ‘~> 1.2.0’pod ‘Firebase/Database’
This will install all the required pods for your application. Apart from the CocoaPods, you will also need to get the API key and enable Cloud Anchors in the Google Cloud dashboard. You can read about how to enable cloud anchors and setting up the Firebase project here. Also, make sure that your Google Cloud Platform billing is up to date and active.
Understanding Cloud Anchors
Before we start implementing cloud anchors, it will be a good idea to understand how cloud anchors work. The position of the anchor does not depend on the latitude and longitude since latitude and longitude are not accurate enough. Latitude and longitudes also does not work indoors.
Cloud anchors begin by mapping the area where you want to drop the anchors. Once the area is mapped the cloud map is uploaded to the Google Cloud platform which then returns a unique cloud identifier. This process is called “hosting an anchor”.
Later, using the cloud identifier along with the approximate cloud map points we can retrieve the anchor. This process is called “resolving an anchor”. We will be using Firebase database to store the cloud identifier along with other information related to the physical object like current color.
NOTE: It is very important that you provide enough mapping information to cloud anchors so it knows about the environment. If you do not provide enough information then your anchors will appear few inches apart as shown in the above video.
Hosting an Anchor
The first step is to initialize the session related to cloud anchors. Create properties for GARSession, ARAnchor ad GARAnchor as shown below. Also don’t forget to add your apiKey for Google Cloud Anchors.
You also need to make sure that you are feeding frames to the GARSession. If you don’t feed frames to the session then cloud anchors will not be able to correctly determine raw points for that area. This can result in obtaining incorrect cloud anchors which may be located few feet apart. This is accomplished in the didUpdate function as shown below:
Next, we will add a cloud anchor where the user touch intersects with the plane.
The addAnchorWithTransform is responsible for hosting the cloud anchor to the Google server.
When the anchor is hosted successfully we get a cloud identifier back as shown in the code below:
NOTE: Cloud anchors have a lifetime of 24 hours.
Resolving Cloud Anchors
Once we have the cloud anchor we can attach virtual objects to those anchors. This process is called resolving cloud anchors. In the code below we used the cloudIdentifier we received after hosting the cloud anchor and used that to resolve the anchors. Once the anchor gets resolved we simply attach a virtual 3D cube to that anchor and add it to the scene view as shown below:
Sharing Cloud Identifier
As you might have guessed, cloud identifier is an integral part of cloud anchors. Using cloud identifier we can retrieve the cloud anchors and then use it to place virtual objects in our surroundings. But how can we share our cloud anchors with other users. For this we need to setup some sort of cloud storage where we can persist our cloud identifiers, along with other attributes of the virtual object that we want to share.
Luckily Firebase Realtime Database is exactly what we need. We can store cloud identifier along with other attributes we want to share in the Firebase Realtime Database which can later be accessed by other users.
The below code shows the implementation of addBlock function which is invoked when the user adds a new virtual object to the scene. The block is added at the cloud anchor position and then later persisted in the Firebase database.
As soon as the record is persisted to the Firebase database an observer is triggered which fetches the cloud identifier and the hex color saved. The code is shown below:
There is lot more code which deals with touching the virtual object and changing the colors of the virtual object. You can download the complete code here.
Google Cloud Anchors is an interested technology that allows to share the AR experience with multiple users. This opens the door for endless possibilities and I can’t wait to see what developers build with it.
If you liked this post and want to support my work then perhaps you will be interested in enrolling in my “Mastering ARKit for iOS” course.