Demonstrates placing and interacting with virtual content in real-world locations using ARDK’s Visual Positioning System (VPS).
Two types of example VPS content are outlined below: bespoke “Tourist” experiences, and “Frost Flower” experiences. An overview of the StreetMap is also provided.
ARDK Features in use
ARMeshManager- Manages the detection of an environmental mesh with accompanying colliders. Used for flower placement in Frost Flower and dynamic shadows in Bespoke.
ARDepthManager- Manages dynamic depth occlusion of virtual content with real-world objects.
ARSemanticSegmentationManager- Used to detect parts of the image containing “sky” and “ground”, then exclude them from depth occlusion.
VPS Managers and Helpers
VpsSceneManager- Handles shared logic across all VPS states and experiences.
VpsIntroManager- Manages the animated airship intro sequence.
VpsDebugManager- Manages debug menu options across VPS states.
PhotoManager- Manages the compositing and saving of the instant photo images.
WayspotAnchorServiceand handles tasks related to localization and placing/resolving anchors.
ICoverageClientand handles querying and storing results from the
VPS Coverage API.
Additional Managers and Helpers
AudioManger- Utility for playing world-space, non-spatial and object relative audio.
ErrorManager- Utility for displaying fullscreen error messages.
A state machine controls the flow of the overall experience. The app can be in one state at a time; each state is authored to dynamically choose the appropriate next state to transition to.
Displays an animated intro sequence featuring Doty’s airship with introductory UI panels.
Following this state, the VpsStreetMap state is sequenced next; see the StreetMap States section.
Displays a GUI with warning text about using AR the first time the user enters AR in a session.
The camera and the AR session are activated along with ARDK meshing. The user is prompted to try to localize by framing a the VPS Wayspot’s “hint” image in the viewport. During the localization process, a feature point visualization is shown to give the user feedback that the process is running, along with hints to help guide the user on how to localize.
StateVpsLocalization uses the
VpsWayspotManager to start its ARDK
WayspotAnchorService and monitors for it to update its
If the localization attempt fails, the user is informed of the localization failure and the system automatically retries. If the localization succeeds, the the state uses the
VpsWayspotManager to restore any
WayspotAnchorPayloads associated with the current
VpsDataEntry in its
VpsWayspotManager uses this lookup to maintain a serialized json file of anchor payload data per localization target identifier. When all anchors restore successfully, the application advances to either StateVpsBespoke or StateVpsFrostFlower, depending on the nature of the VPS wayspot the user selected on the street map. If wayspot anchors fail to restore properly, this is treated as a localization failure that is reported to the user and the system automatically restarts the localization process.
Runs when when the
VpsWayspotManager detects that its
LocalizationState has changed from
Localized. This can happen if the app loses camera tracking or when the app loses focus. All states that display content monitor for the
LocalizationDestabilized event and transition to this state. When localization becomes destabilized, all localized instantiated content is destroyed.
During a particular AR experience, with Doty or Frost Flowers, this state performs the experience (as needed) while displaying a viewfinder and photo button for taking snapshot photos of the experience.
Displays an interactive stack of photos of Doty or Frost Flowers.
Displays an interactive street map showing nearby VPS locations or clusters of VPS locations, built on top of the Mapbox Unity SDK.
This interactive street map shows the user’s current position and marker icons for nearby locations / location clusters. Data for the markers is pulled from the
VPS Coverage API by the
VpsCoverageManager. If no results are returned by the query, the user is given the option to visit a location with known Wayspots.
Coverage API Queries
Each search query through
VpsCoverageManager happens in two parts. First, a call to
RequestCoverageAreasAsync returns a list of
CoverageAreas, each containing one or more
LocalizationTargetIdentifiers. That list of identifiers is then used for a call to
RequestLocalizationTargetsAsync, which returns more detailed information about individual locations (name, coordinates, etc.). Each new location is then wrapped in a
VpsDataEntry, cached and used to draw marker icons on the map.
RequestCoverageAreasAsync for locations that are already cached (from a previous search or as part of an internal list of Frost Flower and Bespoke locations) will be returned directly without the second call to
When sufficiently zoomed-in on the map, marker icons are shown for individual locations. The player can tap an individual location to see information about it, including the location’s hint image which is supplied via the url returned from the
VPS Coverage API. If the player is close enough to localize at the location, they can begin the process of localizing in AR, followed by playing an AR experience at that location. The current version of AR Voyage requires that a user be within 15 meters to localize at a location. However, as GPS is sometimes inaccurate, if the user is within 30 meters of a location for 30 seconds, the app offers an option to “bypass GPS” and advance to localization. It is best practice for a user to be as close to a localization target as possible to increase the chance of a high quality localization. ARDK requires that the user is within 50 meters of a localization target for localization to be possible.
When sufficiently zoomed-out on the map, clusters of locations (within a specific search radius) are shown as marker icons, indicating the type and count of locations they contain.
Clustering is performed based on Geohash, a public domain geocode system.
StreetMap Managers and Helpers
StreetMapManager- Manages functionality related to navigating in the map view.
StreetMapMarkerManager- Manages displaying the user avatar marker, and markers for locations and location clusters.
Activates and directs the interactive StreetMap via method calls to the StreetMapManager.
Following this state, StateVpsLocalization state is typically sequenced next; see the Universal States section.
Bespoke “Tourist” Experiences
Demonstrates custom location-specific content authored for a particular VPS location.
Certain VPS locations in the San Francisco Bay Area have been authored as bespoke “tourist” locations that allow players to interact with and photograph Doty performing location-specific actions.
Data for these locations is stored in scriptable objects that are registered with the
VpsCoverageManager and their custom content is instantiated as part of the localization process by using
WayspotAnchorPayloads that are included in the application’s source.
The authored “tourist” locations are:
Chinatown Gate - Doty poses alongside a lion statue.
Cupid’s Bow - Doty gets in a little archery practice.
Gandhi Monument - Doty arrives in their airship, and makes a flashy entrance.
Pier 14 - Doty casts a fishing line at Pier 14 and makes an unexpected catch.
Willie Mays - Doty swings for the fences in front of the Willie Mays statue.
Instantiates the appropriate custom content based the selected Wayspot.
Displays an animated marker with accompanying arrow to help the player position themselves to view the experience.
Displays a looping animated sequence in which Doty offers the player an instant camera while dynamically following their position.
Following this state, the universal states TakePhoto and ViewPhoto are sequenced, where Doty performs their location-specific custom behaviors.
Displays the “Tourist” badge unlock message.
Doty plants a flag at this location and then departs.
Frost Flower Experiences
Demonstrates saving and restoring content placed dynamically by the user at a generic VPS location.
For generic VPS locations (i.e., the bulk of locations that don’t have “Bespoke” tourist content), players are given the opportunity to plant seeds that grow into “Frost Flowers”. Flowers grown during the first visit will soon blossom into larger “Superblooms”, viewable on a subsequent visit to the same location.
After localizing, a player can plant seeds by targeting a reticle on an invisible
AR Mesh object (provided by
ARMeshManager) and pressing a launch button. On impact, the
VpsWayspotManager is called to place a
WayspotAnchor using the transform for an instantiated “hero”. Once the anchor is created, the hero flower’s transform is automatically childed to the anchor so that it will receive any anchor pose updates that come from ARDK. A secondary process then begins raycasting toward the
AR Mesh to find growth points for additional plants and grass.
Upon successfully planting a third seed, the
WayspotAnchorPayload for each flower is serialized and stored locally on the device.
On subsequent visits to a given location, existing
WayspotAnchors are deserialized and resolved as part of the localization process described above. New “hero” flowers and secondary foliage are then regrown at their positions.
FrostFlowerManager handles most aspects of targeting and planting seeds and all associated animation and visual effects. It is also responsible for handling the save/load of frost flower data between user visits to a given location.
Displays an interface for launching seeds and growing Frost Flowers when initially visiting a location. On subsequent visits the planted seeds are shown with larger blooms and more secondary foliage.
Following this state, if this a subsequent visit to this location, the universal states TakePhoto and ViewPhoto are sequenced next so the user can take a photo of the now “harvested” Frost Flower “Superblooms”.
Displays the “Superbloom” badge unlocked message.
Displays a final success message.