[vc_column width="1/6"][/vc_column][vc_column width="2/3"][vc_text_separator title=""][vc_column_text]

Holo·AR·t

Holographic Art

Judith Amores, Jaron Lanier. 8th Feb'17 

HoloARt is a new media art that explores the use of holograms in mixed reality for creative self expression. We designed a system that allows the user to turn their physical environment into a canvas where digital holograms and physical objects co-exist in the real and virtual world. Users are able to virtually spray and splatter hologram paint on top of physical objects and surfaces as well as painting in the air by only using their hands. The content grows dynamically, following the natural movements of the user. The system is self-contained and does not require hand controllers nor positional tracking sensors on the space.

---

[/vc_column_text][vc_video link="https://player.vimeo.com/video/202652308" css=".vc_custom_1486501892034{margin-right: 0px !important;margin-left: 0px !important;border-top-width: 0px !important;padding-bottom: 0px !important;}"][vc_text_separator title=""][vc_column_text]

HoloARt is the first Mixed Reality system that allows the user to draw, spray and splatter digital paint anywhere in the "real" physical space.

---

[/vc_column_text][vc_column_text]

Immersion is commonly used among artists to reach to that special state that we call "flow". A feeling outside of time that artists use to dive in the creative process. This makes Mixed Reality an especially interesting medium for art generation because it brings a level of escapism unlike any other. In the world of art the medium is defined as the material or tool that the art work is made of; such as clay in the case of ceramics. In HoloARt we can see the medium as a digital hologram. The content grows with the movements of the user’s hand and co-exists with the physical environment. HoloARt is the first Mixed Reality system that allows the user to draw, spray and splatter digital paint anywhere in the "real" physical space. We create a spatial mapping of the physical space so the user can move around and use any object, surface or the air as a canvas. HoloARt consists on three different techniques or "sub-mediums": "Air Paint", "Graffiti" and "Splatter". The user is also able to Edit, Erase or choose the Color of the art work. The interaction techniques and design principles we defined for HoloARt are the result of an exploration inspired by artistic techniques used by painters, photographers and designers.

---

[/vc_column_text][vc_column_text]

"air painting": LIGHT PAINTING IN REAL TIME

---

[/vc_column_text][vc_single_image image="3490" img_size="full" add_caption="yes" alignment="center" css=".vc_custom_1486449851985{margin-top: 0px !important;border-top-width: 0px !important;padding-top: 0px !important;}"][vc_column_text]

HoloARt. On the left, the user is "Air Painting" using her index finger. On the right, a mannequin has sprayed paint that matches its 3 dimensional shape.

---

[/vc_column_text][vc_column_text]

The aesthetics and interaction used for "Air Painting" resemble the photographic technique of "Light Painting". Transparencies and superimposition of the textures on top of the physical space can easily be achieved by moving the index finger around the physical space. Users draw using the tip of their index finger and air tap to stop or start the drawing. The results vary among users, the drawings are usually not logically structured, but it is rather a result of the movements and "flow" of the person.

---

[/vc_column_text][vc_single_image image="3484" img_size="full" alignment="center"][vc_column_text]

Picasso's "light drawings," series (1949) LIFE magazine.

---

[/vc_column_text][vc_column_text]

The first artists who explored the idea of painting in the air or "Light Painting" were Man Ray in his series Space Writing, Gjon Mili and Pablo Picasso, who in 1949 published his series of photos Light drawings (see image). Such drawings are captured by the camera, however they are not perceivable by the user in real-time. HoloARt  allows the user to create such paintings in real-time using mixed reality with no need of external light.

---

[/vc_column_text][vc_column_text]

"GRAFFITI": change the color of objects & surfaces 

---

[/vc_column_text][vc_single_image image="3518" img_size="full" alignment="center"][vc_column_text]

Graffiti or Surface Painting lets the user spray on objects, walls and any type of surface. Users can spray paint by holding the thumb and index finger together (pinch gesture) or holding the index finger down. A target cursor follows user’s hand position and is projected at the predicted destination of the sprayed paint. The distance between the hand and the surface it is directly proportional to the size of the spray and inversely proportional to the opacity of the paint. Therefore, if the distance between the hand and the surface increases; the size of the spray increases and the amount of paint is distributed. The result is sprayed paint with bigger size but lower degree of opacity. On the contrary, if the hand is closer to the surface, the spray will be smaller but with higher opacity (see image). These type of hand gestures and spray have been designed to mimic real graffiti techniques that artists use.

[/vc_column_text][vc_single_image image="3519" img_size="full" alignment="center"][vc_column_text]

---

"SPLATTER": Splash paint with real physics

---

[/vc_column_text][vc_column_text]

Users can splatter paint by pointing with inertia towards surfaces. The size of the splat is proportional to the velocity of the user’s movement and is affected by gravity. If the user throws paint with more force, the splatter will be bigger (mimicking the technique used by artist when splattering paint with a brush). If the user splats paint on the ceilings, some of the paint will fall down (see image).

[/vc_column_text][vc_single_image image="3527" img_size="full" alignment="center"][vc_column_text]

---

EDIT, ERASE And change colors or textures

---

[/vc_column_text][vc_column_text]

Aside of the different mediums we described above, we created some basic tools such as Edit, Erase and Color. The user can delete paintings by hovering an eraser on the painting while a sound feedback and particle explosion happens. Edit lets the users grab and move paintings. The user can grab a painting by pinching it and then move it while holding the gesture. A subtle change on the opacity provides feedback to the user when the painting is selected. Finally, the Color tool lets the user pick the color of the painting. We experimented with 3D and 2D color pickers (see images bellow).

[/vc_column_text][vc_single_image image="3525" img_size="full" alignment="center"][vc_column_text]

---

Human augmentation: voice-command bracelets as menus

---

[/vc_column_text][vc_single_image image="3537" img_size="full" alignment="center"][vc_column_text]

We designed a bracelet-menu that augments the user’s forearm and provides an intuitive interaction technique that does not depend on a controller. The menu has a hierarchical layered interface that is designed so as to fit on top of the forearm as if it was a set of bracelets (see image bellow). The design of the menu has been placed strategically on the forearm so as not to be seen while the user is painting. Because of the limited field of view provided by the Hololens, the bracelet is not displayed when the user paints, splatters or sprays paint. In this manner, the user has both the convenience of an easy access to the menu by just looking slightly down to the forearm and the advantage of not being distracted if painting. For the same reason we choose voice commands for selecting tools and brushes.

[/vc_column_text][/vc_column][vc_column width="1/6"][/vc_column]

[vc_column width="1/6"][/vc_column][vc_column width="2/3"][vc_column_text]

Technology has broadened the scope of New Media Art and has opened the door to new forms of artistic expression and Human Computer Interaction such as holoart.

---

[/vc_column_text][vc_text_separator title="Publications & Exhibitions"]

[vc_text_separator title="Behind the scenes"][vc_column_text]

HoloARt is the result of an internship (June-August) at Microsoft Research. Jaron Lanier was my mentor over the summer, who greatly guided me with the overall idea and interactions of the system. The idea and implementation was executed from mid June to mid August. At the end of August we wrote 2 patents, created a video of the project, wrote a paper for the CHI Conference and demoed to VP of Microsoft Research, the HoloLens Design Team, directors of Hololens, etc.

[/vc_column_text][vc_text_separator title="From Research to Product"][vc_column_text]

HoloARt has evolved from the research project described above into an official Hololens app called "Holographic Art". You can download in the store.

[/vc_column_text][vc_tweetmeme type="hashtag" hashtag_no_default="" hashtag_custom_tweet_text="#HoloARt is a mixed reality creative platform that lets you paint the physical environment using ``holographic” ink @MSFTResearch @jdthamores" hashtag_recommend_1="@MIT" hashtag_recommend_2="@medialab" css=".vc_custom_1486584186893{background-color: #ffffff !important;}"][vc_text_separator title="holoart.media.mit.edu"][/vc_column][vc_column width="1/6"][/vc_column]

[vc_column][vc_separator][/vc_column]