Editing

The audiovisual Augmented Reality experience of lifeClipper2, the predecessor of this project, was created by a camera fixed to a headset. The camera captured and represented what the visitor would be seeing in the real world, and produced a real-time video stream, which was then applied to a virtual background plane fixed in the visitor’s viewing direction. Virtual content (3D bodies, textures, lights, etc.) were placed in front of that video backdrop. The reality plane therefore hosted all the virtual extensions on a flat background image (see figure below).

Figure left: 3D model with the luma-keyed skyline around the terrain.
Figure right: Augmented Reality projection system with video backdrop and 3D model of terrain – both textured with the real-time video stream. The worm-like creature intersects the terrain model and casts a shadow onto it.

In lifeClipper3 the same hardware system is used, but the projection plane for the real-time video stream is only used for the park's skyline and transparency effects in the sky. The main part of the video is projected directly onto the ground model of the park itself, using the “planar mapping” technique. With this approach, the projected pixels of the video image coincide with the spatial representation in virtual space, which overlays reality at a scale of 1:1.

Although direct projection onto the organically shaped 3D body creates shadows and the calibration inaccuracies have to be hidden by semi-transparent layering, the new system offers a set of crucial possibilities:

It allows a more intuitive placement of 3D models in relation to the camera image of reality; 3D bodies can intersect the real terrain and cast shadows on it; virtual lights can throw light onto the video textured terrain; and ambient effects, like fog, can be applied gradually in depth. Additionally, by changing the texture of the real-time video stream to any other given texture, the state of the terrain can be altered, so that, for example, the park can be converted into a desert. In lifeClipper3 the dominance of the real or virtual world constantly shifts, depending on climate changes.