Prototyping at Factory 42

For the Factory 42 development team, the greatest feature of using Magic Leap – the element that pushes it ahead of any other augmented or mixed reality technology to date – is its ability to combine the physical and digital worlds together. The simple ability for digital content to react to physical objects in the real space: a table, the floor, even the player’s own hands, is a revolution in realism and interactivity. This concept is continuously at the heart of our designs and the prototypes we are developing.

For a few months now, the development team have been exploring the various functions and features of the Magic Leap One devices. Since then, we’ve put together prototypes and demos that test the breadth of Magic Leap’s features, including image tracking, hand and gesture recognition, eye tracking, telemetry and world reconstruction to name a few. We’ve even built our own online multiplayer scenes in which players can all wear Magic Leap headsets and share the same experience together.

Presented in this post are a handful of some of the features that are available with Magic Leap.

World Reconstruction

In order for our digital content to interact with the world around us, the Magic Leap device needs to scan its surroundings and build up a mesh of the environment; this is called World Reconstruction.

Once Magic Leap has created a digital version of the world we can experience true mixed reality.

Magic Leap One user's view of scanning of environment with mesh overlaid on office

Magic Leap Create app scanning environment in app setup
Create by Magic Leap

Image Tracking

The ability to recognise a specific image, and follow its position around the room, is a feature common to many mobile AR apps. Image tracking can be a useful way to activate your digital content at key locations in your scene; they can make pictures come to life.

View through Magic Leap headset showing tracking of an image on a desk

Important design notes to consider when using Image Tracking:

  • Image Tracking typically works best if the image is stationary and of a decent size; we don’t often go smaller than A5 if we can help it.
  • When deciding on which pictures to use, bear in mind that the Magic Leap image tracker discards all colour information, so try to choose images with high contrast.
  • Glossy or reflective finishes will interfere with detection of your image.
  • Tracking works best when the image is flat. Granted Magic Leap does not need to detect the entire image for recognition to occur, but bending or distorting your image around a 3D surface will increase the likelihood of errors.

Hand Recognition and Gestures

Hand Tracking, similar to Image Tracking, is the recognition of key points on the user’s hands. In addition to tracking all five fingers on both hands, Magic Leap is able to detect a number of specific hand gestures made by the player. These include: Open Hand, Closed Fist, Thumbs Up, Pointing, L shape, C shape, Pinching and OK.

Drawings of the eight hand gestures tracked by the Magic Leap One headset
Source: Magic Leap Creators Portal (June 2019)

View through Magic Leap One headset showing handtracking overlaid on user's hand

 

Leave a comment

Your email address will not be published. Required fields are marked *