Our basic idea is to use computer vision techniques, in real time, to dynamically extract per-pixel depth and reflectance information for the visible surfaces in the Office, such as walls, furniture, objects and people.

Using this depth information one could then display (project) images on the surfaces, render images of captured models of the surfaces, or interpret changes in the surfaces. We are primarily concentrating on the display and capture aspects.

Our approach to the former is to designate everyday, irregular surfaces in a person’s office to be part of a spatially immersive display, and then project high resolution graphics and text onto those surfaces.

Our approach to the latter is to capture, reconstruct, and then transmit dynamic image-based models over a network for display at a remote site.   More detail...