Sponsors / Partners

Accenture Microsoft NYSE Shell Salesforce AAAS Science Red Herring Fortune CNN Time Magazine

2004 World Technology Awards Winners & Finalists

Dr. Adrian David Cheok and Dr. ZhiYing Zhou

Please describe the work that you are doing that you consider to be the most innovative and of the greatest likely long-term significance.

MAGIC CUBES FOR MIXED REALITY EDUCATION http://mixedreality.nus.edu.sg/RESEARCH/MC/MC.webpage/research-MC-infor.htm

An invention for mixed reality family education has been developed by researchers in Singapore. Mixed reality technology, in which computer graphics are superimposed on the real world, enables traditional "magic cubes" to have three-dimensional fantasy for family entertainment. It has just been filed in 5 separate US Patents.

A standard magic cube is made up of a handful of small wooden or plastic cubes connected at various surfaces or edges. These can be unfolded or combined in many ways to reveal a sequence of puzzles or different pictures. Now Professor Adrian David Cheok and research fellow ZhiYing Zhou at the Mixed Reality Lab at the National University have updated this kids' toy and endued it with computer graphical magic.

Their first attempt is to let the "magic cube" tell the Old Testament story of Noah's ark. The computer can tell the story of different stages which the user reveals by unfolding the cube. To watch the 3D story scenes, the user wears a head mounted display with a small camera attached to the front which is connected to software and is able to recognizes the symbols printed on different stages. At each stage, the computer uses the symbols as its cue to run a segment of audio and animated three-dimensional video, which tells part of the Noah's ark story. To facilitate ease of use, virtual animated arrows as well as physical arrows printed on the cube tell the user how to unfold the cube to move to the next scene. In early lab tests, children said they enjoyed the Magic Story Cube more than a picture book. "You need to take account of the fact that people are physical creatures, and these cubes are a very tangible, direct way of interacting with information," Cheok says.

Their second attempt is to use the physical cubes as the essential parts of the computer games. Cubes are used as a dice and control tool of a monopoly-style board game, called "Jumanji Singapore". The purpose of this game is for the users to have a virtual tour of the three-dimensional Singapore attractions while having fun with the board game competition. By wearing the head mounted display, the players can see three-dimensional Singapore map pop-up on the physical game board. They can also see their metaphors walking on the tracks of the virtual Singapore map after rolling the dice. The game tour starts and ends at the Changi airport. Along the game tracks, the players may stop at Singapore attractions like City Hall and Sentosa. Corresponding photos and videos are played back by manipulating the physical cube when the player gets to a specific spot. The player can even fly into and navigate the virtual reality world by moving the control cube around. The player can also go shopping by using the cubes to put goods into the cart. "The whole point is to make traditional computer games tangible and allow face to face social interactions. Just like the traditional physical board games which we enjoy since our childhood," Cheok says.

Magic Cubes More Pictures

The new form of family entertainment system pioneered by Magic Cubes has received recognitions at various levels. Recently, Magic Cubes representing NUS has been awarded the top international prize at the Fifth National University Student Contest of Creativity-in-Action and Asian University Creativity Tour. In June 2004, Washington DC, Magic Cubes is the only accepted demo from Asia to be represented at the premiere ACM SIGCHI International Conference for Interaction Design & Children (IDC2004). Five US patents have been filed on the system and potential applications for education and family entertainment. The biggest Singaporean company, Creative Technology, has decided to cooperate with NUS to make products from these patents for family entertainment market. "This will definitely beloved by the grandparents for playing together with the kids," says a senior officer of Creative Technology.

Brief Biography

Adrian David Cheok is aiming to push the boundaries of research into interactive new media technologies through the combination of technology, art, and creativity. He developed Mixed Reality technologies to superimpose output from a 3D virtual viewpoint camera system over the real world. Users see life-sized live 3D virtual images of remote collaborators standing in the real world in front of them. This is the first time that this technology has been demonstrated, enabling him to conduct ground-breaking user studies to evaluate Mixed Reality conferencing, and also to develop unique tools and interaction techniques for supporting remote collaboration. He also developed ground breaking new forms of mixed reality computer entertainment, such as the "Human Pacman" project. The technology opens up exciting new opportunities in the areas of computer-graphics and human-computer-interface development. There will be applications in a great variety of areas such as education, architecture, military, medicine, training, sports, computer games, tourism, video conferencing, entertainment, and human welfare.

He has produced large scale technological deliverables for DSTA and the Singapore military in interactive human computer systems. He has spun off two companies in 2003, Real Space and Brooklyn-Media. Real Space was funded by Mr. Sim Wong Hoo (CEO Creative Technologies) and is producing mixed reality systems for military and industry. Brooklyn-media is commercializing systems for digital games and mobile entertainment using novel interactive technologies. He has also been invited and have exhibits in major media technology-art centers, such as a two year exhibit in Ars Electronica Museum of the Future in Austria. Apart from academic journals, the work has been featured in numerous international media including CNN and CNBC, newspapers in Germany and Austria, and Spanish television. In the near future we are doing a project together with a Vienna company for the 250th commemoration of Mozart, which is funded by Vienna city council as the first prize winner work to create a Mozart-Magic-Book-project. His work on Human Pacman was demonstrated at CHI Vienna, 2004.

Zhiying Zhou is a Research Fellow with the Department of Electrical and Computer Engineering, National University of Singapore. His research interests include augmented and virtual reality, computer vision, speech signal processing, tangible user interfaces, and multimodal human–computer interaction. Zhou received a B.Eng and a M.Eng in Instrument Science & Engineering from Southeast University, Nanjing, China. His PhD thesis was submitted to National University of Singapore in May, 2004.