This is part six of a nine-part series that explores how to create a Unity application for a mobile device, using Augmented Reality and Azure Spatial Anchors

• In Part 1 we introduce the project, requirements, technologies and various AR concepts.
• In Part 2 we follow the steps in the project QuickStart guide.
• In Part 3 we create a basic Unity ARFoundation application.
• In Part 4 we look at one of the building blocks of AR - the “Placement Cursor”.
• In Part 5 we explore how the UI for the project works.
• In Part 6 we learn how Azure Spatial Anchors can be used in our project.
• In Part 7 we step up our game by looking at how we can add and move virtual scenery.
• In Part 8 we develop functionality that saves and restores virtual scenery to the cloud.
• In Part 9 we learn about the Azure Functions backend for the solution.

The entire project sourcecode that accompanies this series can be found at:

• GitHub : SiliconOrchid : Unity-AR-AzureSpatialAnchor-Demo

In this article, we look at the core of this demonstration - working with spatial anchors.

• The primary demo script for working with ASA can be found in Assets/Scripts/Managers/CloudAnchorSessionManager
  • View in GitHub
• The GameObject which instances the Component is /AppManagerStack

Spatial anchors

When we talk about spatial anchors, we’re actually addressing two separate concerns (although they do overlap):

• Native Anchor - (aka “Reference Point”).

  • This is provided by AR Foundation (which in turn, builds upon ARKit and ARCore).
    
  • With a Native Anchor we can define a single point in 3D space which will correlate to the real world.
    
  • A native anchor is self-contained within the app and requires no networking to function.
  • The Unity Component related to this is Packages/AR Foundation/Runtime/AR/ARReferencePointManager. In this demo project, it can be found as part of the GameObject “ARFoundationStack”

• Azure Spatial Anchor - (aka “Cloud Anchor”).

  • The ASA SDK provides an extension onto a Native Anchor.
    
  • This builds upon all of the preexisting native functionality, such as sparse-point-clouds - but then seamlessly blends in features such as connectivity to the companion cloud service and point-cloud matching of previously saved anchors.
  • The Unity Component related to this is Assets/VendorAssets/AzureSpatialAnchors.SDK/Scripts/SpatialAnchorManager. In this demo project, it can be found as part of the GameObject “AppManagerStack”

Overview of the process

The general process for creating and saving an anchor is as follows:

• we use our “placement cursor” to identify a point in 3D space where we want our anchor to be placed.
• a GameObject that will represent the anchor is spawned at the coordinates.
• spatial anchor components are associated with the GameObject using the ASA SDK. This takes care of making the anchor both a cloud and native anchor.
• an ASA SDK method is called which saves the anchor to the cloud - when it has enough point-cloud data.
  
  • If there is not currently enough data, this is reported by the SDK and we can use this in the UI to prompt the user to capture more data.
  • we can define a lifespan of the anchor record however we choose.
• the SDK returns an object which is identified with a unique ID.
• to be able to search for this anchor, we will need to persist this cloud anchor ID somehow
  • in this demo, we use our own cloud API backed by cloud storage for persistence between AR sessions.

The general process for restoring a previously created anchor is as follows:

• we place an API call to our own cloud service, which will return the most recently-set anchor ID.
• upon receipt of that anchor ID, we then use the ASA SDK to query for that particular anchor by its ID.
• a “watcher” is created, along with any necessary criteria, which processes the camera feed, looking for matching sparse-point-cloud data.
• when a match is detected, a method can be triggered - in this project, we use that opportunity to re-create an anchor GameObject which provides a visualisation to the user

Starting a spatial anchors session

A spatial anchor session:

• is created in our startup method
• then exists for the lifecycle of the application.
  
• represents a union between a native AR session and a network connection to the ASA cloud service.

In our project, the code for starting up a session can mostly be found in the method SetupCloudSessionAsync() and can be distilled down to the following abbreviated code:

private SpatialAnchorManager spatialAnchorManager;

void Awake()
{
    spatialAnchorManager = FindObjectOfType<SpatialAnchorManager>();
    SetupCloudSessionAsync();
}

private async void SetupCloudSessionAsync()
{
    await spatialAnchorManager.CreateSessionAsync();
    await spatialAnchorManager.StartSessionAsync();
}

Note: the startup process also performs a few other initialisation activities, such as setting the UI state, etc.

Saving a spatial anchor to the cloud

• The method SaveAnchor() is the entry-point into this functionality.

  • It is called by the UIButton_SetAnchor_Click() button-click handler in the UIManager Component.
    
  • Because the UI button is actually used for two purposes (first creating an anchor - and then saving the anchor), the code looks at the application state value appStateManager.currentCloudAnchorState to inform us which task we should be doing.

• The method SpawnNewAnchoredObject() performs a number of tasks:

  • Spawns an instance of the “SpatialAnchorGroup” Prefab. This GameObject represents both:
    • a “visible representation” of the anchor (nothing more than a simple sphere)
    • an object that will be used as a container for “scenery” objects (we’ll talk about this properly, later in the series).
  • Attaches an ASA CloudNativeAnchor Component to our GameObject.
    
    • According to the class comment, this is “a behavior that helps keep platform native anchors in sync with a CloudSpatialAnchor”
  • Calls the CloudToNative() method
    • According to the method comment, this “Stores the specified cloud version of the anchor and creates or updates the native anchor to match.”

As a reminder, ASA is still a preview technology;

Although I haven’t personally experienced this problem, at the time of writing, I note that there are bugs related to the CloudToNative() method that are currently being addressed by the team at MS - refer to GitHub Issues : CloudToNative() does not work with localized anchors for more information.

• The method SaveCurrentObjectAnchorToCloudAsync() calls the ASA SDK methods related to storing the current anchor to the cloud.

  • Our project hard-codes the expiration time of our cloud anchors to two days. This setting can be found in the line:

  cloudSpatialAnchor.Expiration = DateTimeOffset.Now.AddDays(2);
  

  
  

  • before attempting to save to the cloud, the spatialAnchorManager makes sure there is enough sparse-point data available:

  while (!spatialAnchorManager.IsReadyForCreate)
  {
  await Task.Delay(200);
  float createProgress = spatialAnchorManager.SessionStatus.RecommendedForCreateProgress;
  appStateManager.currentOutputMessage = $"Move your device to capture more data points in the environment : {createProgress:0%}";
  }
  

  
  

  • saving of the anchor to the cloud is performed with these lines:

  await spatialAnchorManager.CreateAnchorAsync(cloudSpatialAnchor);
  currentCloudSpatialAnchor = cloudSpatialAnchor;
  

  
  

  • finally, we can specify a method to be called when the anchor has been saved to the cloud. We’ll talk about that follow-up activity in the next section below:

  if (currentCloudSpatialAnchor != null)
  OnSaveCloudAnchorSuccessful();
  

Saving a spatial anchor ID to our own API service

When we call the ASA SDK method CreateAnchorAsync(), the cloud service returns a CloudSpatialAnchor result that:

• is copied to the private field currentCloudSpatialAnchor.
• contains the cloud anchor identifier ID.

We can then use this ID, in another AR session (either in the future - or on another device), to query the ASA service for that specific spatial anchor.

We’ll look at our implementation of an HTTP API cloud service that persists the latest cloud anchor ID, separately in Part 8 - Azure Functions Backend.

In this section we are interested in making an HTTP client call from Unity, to POST data to that service:

private void OnSaveCloudAnchorSuccessful()
{
    StartCoroutine(SendAnchorIdToCloud( currentCloudSpatialAnchor.Identifier));
}

The Coroutine uses the built-in Unity HttpClient UnityWebRequest to POST the ID string to the API.

IEnumerator SendAnchorIdToCloud(string anchorId)
{
    using (UnityWebRequest unityWebRequest = UnityWebRequest.Post(generalConfiguration.apiUrl_CloudAnchorId, anchorId))
    {
        unityWebRequest.SetRequestHeader("Content-Type", "application/json");
        yield return unityWebRequest.SendWebRequest();
    }
    OnAnchorIdSavedSuccess();
}

OnAnchorIdSavedSuccess() is simply used to update application state and create an onscreen status message.

Note: The fact that this is a POST and not a GET is relevant, because our backend also has a second API, using the same same URL, for retrieving the last used ID with a GET request:

Restoring a spatial anchor ID using our API service

The process of searching for the last previous saved anchor is largely the same process in reverse.

• The method RestoreAnchor() is the entry-point into this functionality.
  
  • It is called by the UIButton_RestoreAnchor_Click() button-click handler in the UIManager Component.

    public void RestoreAnchor()
    {
        if (appStateManager.currentCloudAnchorState != CloudAnchorStateEnum.ReadyToLookForCloudAnchor)
        {
            appStateManager.currentOutputMessage = $"Retrieving last used anchor ID from cloud...";
            appStateManager.currentCloudAnchorState = CloudAnchorStateEnum.ReadyToLookForCloudAnchor;

            StartCoroutine(GetAnchorIdFromCloud());
        }
    }

• The Coroutine GetAnchorIdFromCloud() uses the UnityWebRequest HTTP client to make a GET request to retrieve the ID string value.

    IEnumerator GetAnchorIdFromCloud()
    {
        UnityWebRequest uwr = UnityWebRequest.Get(generalConfiguration.apiUrl_CloudAnchorId);
        yield return uwr.SendWebRequest();

        if (uwr.isNetworkError)
        {
            appStateManager.currentOutputMessage = $"Failed retrieving last used anchor ID from cloud.";
        }
        else
        {
            appStateManager.currentOutputMessage = $"Retrieved last used anchor ID from cloud, OK.  Looking for anchor - move the device about...";
            currentAnchorId = uwr.downloadHandler.text.Trim();
            SetAnchorIdsToLocate();
            currentWatcher = CreateWatcher();
        }
    }

• The method SetAnchorIdsToLocate() sets up the “Anchor Locate Criteria” mechanism.
  • Technically, the SDK provides us with the option to specify a collection of multiple anchors to look for … but in this demo, we’ll only be looking for a single anchor.

    private void SetAnchorIdsToLocate()
    {
        anchorIdsToLocate.Clear();
        anchorIdsToLocate.Add(currentAnchorId);
        anchorLocateCriteria.Identifiers = anchorIdsToLocate.ToArray();
    }

• When an anchor is detected, the method CloudManagerAnchorLocated() is triggered.
  • Amongst the data made available to us is a Pose which provides the coordinates for the anchor (relative to the session start)

    private void CloudManagerAnchorLocated(object sender, AnchorLocatedEventArgs args)
    {
        if (args.Status == LocateAnchorStatus.Located && appStateManager.currentCloudAnchorState == CloudAnchorStateEnum.ReadyToLookForCloudAnchor)
        {
            if (spawnedAnchorObject == null)
            {
                currentCloudSpatialAnchor = args.Anchor;
                Pose anchorPose = currentCloudSpatialAnchor.GetPose();
                spawnedAnchorObject = SpawnNewAnchoredObject(anchorPose.position, anchorPose.rotation);
            }

            appStateManager.currentCloudAnchorState = CloudAnchorStateEnum.NothingHappening;
            appStateManager.currentUIState = UIStateEnum.SceneryButtonsOnly_NothingHappening;
            appStateManager.currentOutputMessage = $"Found cloud anchor OK.  Press 'Place' to add new scenery, or press 'Restore' to get previously saved scenery..";
        }
    }

Next, in part seven, we step up our game by looking at how we can add and move virtual scenery.

NEXT: Read part 7