using avatars to interact with PHYSICAL objects

[/vc_column_text][vc_text_separator title="MIT MEDIALAB, LA SALLE URL BARCELONA"][/vc_column]

SmartAvatars is a system that is based on a mixed reality environment containing virtual avatars as a medium for the interaction between the user and smart objects in the environment.

[/vc_column_text][vc_video link="https://player.vimeo.com/74210099" css=".vc_custom_1449291723362{margin-right: 0px !important;margin-left: 0px !important;border-top-width: 0px !important;padding-bottom: 50px !important;}"][vc_single_image image="3065" img_size="full" alignment="center"][vc_text_separator title="Qbox"][vc_column_text]

Qbox is a magical box which brings toys to life. When a small toy is put inside the box, the child-user will see it come to life through an AR interface. The toy will interact with the box, with the kid and also with the world. Our idea was to develop an entertainment and educational product for children.

We used an Arduino Mega as a microcontroller and Wifi connection to send and get the information, moreover we used RFID to detect the different types of figures inserted in the cube. For the wheels we used two servos. We implemented also capacitive sensors around the cube walls to detect where are you touching the cube. We used Unity, Vuforia and 3dmax to display the augmented reality content. You can also check out our first prototype.

[/vc_column_text][vc_single_image image="3066" img_size="full" alignment="center"][vc_text_separator title="Qbox"]

As part of the overall concept, one of the prototypes is called "Flexo". With Flexo we investigate the virtues and obstacles that occur when we use an augmented reality (AR) avatar to interact with a simple smart object. In order to develop the prototype, we hacked a lamp in order to be recognized by a smartphone since the device would display augmented reality content. We created a 3D animated avatar using Motion Capture and throughout Unity we unified the 3D content with the augmented reality. In this project I've worked on the original idea, the software and movements of the avatar and the modeling and animation of the character.

[/vc_column_text][vc_video link="https://player.vimeo.com/84610684" css=".vc_custom_1449290857216{margin-right: 0px !important;margin-left: 0px !important;border-top-width: 0px !important;padding-bottom: 50px !important;}"][vc_video link="https://player.vimeo.com/video/84610711" css=".vc_custom_1449294368982{margin-right: 0px !important;margin-left: 0px !important;border-top-width: 50px !important;padding-bottom: 50px !important;}"][vc_text_separator title="Augmented Waiter"][vc_column_text]

This platform was developed as a part of THOFU project (Technologies for the Hotel of the Future, Government of Spain). We displayed a virtual waiter that offers you the daily menu and interact with the user. The waiter has automatic body gestures, facial expressions and a head look controller. This way, the waiter looks like a real person. The system is able to generate automatic gestures and lip synchronization between the avatar and the audio input. In this project I've worked on the idea, the integration between BodySpeech and the augmented reality avatar in a iOS environment, and the lip sync system. I used 3dmax for the design of the avatar. I used Unity, Vuforia and Tagarela for the integration between the lip sync-body gesture system and the augmented reality. I used C++ and the SAPI Lipsync open source software for developing the lip sync system.

[/vc_column_text][vc_video link="https://player.vimeo.com/video/75124686" css=".vc_custom_1449294685103{margin-right: 0px !important;margin-left: 0px !important;border-top-width: 50px !important;padding-bottom: 50px !important;}"][/vc_column][vc_column width="1/6"][/vc_column]