For about two years, I was part of a mega talented incubation team that prototyped new and challenging experiences on the HoloLens platform. Two titles that shipped with the hardware were Fragments and Holotour. Here are a few details about the work I did on those titles.
FRAGMENTS
Fragments is a mixed reality game where players become detectives in a high-tech crime thriller, where you follow the clues to solve a mystery that takes place in your real-world space.
Here are a few time stamps from tools, where I focused a ton of my efforts:
Sound Tool
- https://youtu.be/XWFYdEyvBVk?t=980
- Example of the Sound tool, where the player “collects” 3D positional sound evidence
IR Heat Tool
- https://youtu.be/XWFYdEyvBVk?t=1447
- Example of the IR Heat tool, where the player can use heat signatures to gain information from the crime scene
UV Tool (& Reconstruction)
- https://youtu.be/XWFYdEyvBVk?t=2020
- Example of the UVtool, where the player can see otherwise invisible element to gain information and evidence from the crime scene
You can jump around the video to see other aspects of the MR experience. I can speak to the playback aspect of the animation (the crime), evidence and the evidence locker, clues, solving, etc. It was a fun game to work on.
HOLOTOUR
HoloTour combines 360° video, spatial sound, and holographic scenery as the foundations of a virtual travel tour app that you navigate.
I primarily worked on front end UI, the passport, and the entirety of the Peru Tour. Every 3D POI is an interactive with VO and sometimes with an animation or MR object attached to it. The app was super highly rated, and educational too. There’s too much to share, I wish there was a better video of the entire tour, but, here are a few clips of some of the cool stuff we did:
Ollantaytambo
- https://youtu.be/Ql5aWF0vURc?t=120
- Battle of Ollantaytambo, where the user can see a reenactment of the flooding of the valley by the Inca vs the Spanish
The Passport
- https://youtu.be/Ql5aWF0vURc?t=334
- The Passport allowed the user to rate the tour and track their progress (we used this for telemetry as well)
The Hot Air Balloon over Machu Picchu
- https://youtu.be/Ql5aWF0vURc?t=888
- The Hot Air balloon used a 360 pano that we built a MR balloon that the user stood upon.
- It has a working telescope that zoomed in on one of the POIs and a couple of other pretty cool wow moments.
Milky Way over Machu Picchu
- https://youtu.be/Ql5aWF0vURc?t=1239
- We took a day pano and wanted to show the night sky and a few constellations, so we superimposed the nighttime view from the position the user was standing and ducked out the day pano.
Mark Simon 