A versatile way to communicate emotions, emojis, and compressed ideas in virtual reality.
In my undergraduate design course that examines the future of interaction design in mixed reality, our group was tasked with exploring how everyday interactions with mobile devices can be transformed into magical/delightful extended reality (XR) experiences.
Inspired by Gen-Z culture and emojis, we designed a gestural menu in VR that allows people to create, store, and access fun visual expressions and objects, and use them to communicate with others in virtual realty.
Through our initial exploration of communication in XR environments, we chose to investigate how typing in VR could benefit from different physical gestures. After trying VR typing ourselves, we learned that aiming and clicking on individual letters felt tedious and slow. When we began brainstorming keyboard and hardware redesigns to solve this problem, our mentors advised us to refocus our efforts on spatial interactions in XR instead.
We brainstormed different types of text and digital communication, categorized them, and created different types of scenarios for them. One category, short messages and emojis, piqued our interest.
Brainstorming reasons why someone would use emojis in XR over smartphones revealed two interesting insights. Since people can inhabit the same space in XR, it allows them to see each other react to emojis in real time. Also, emojis could have much greater visual impact in VR, given that they could be presented in 3D space.
Our goal for this project was to communicate the magic of sending and receiving exciting emoji effects in VR.
Users of our tool need to bring up emojis in XR just as fast as on a keyboard, and through sketching and brainstorming sessions we based our emoji system off of video game emote systems. We were specifically inspired by online multiplayer video games because they usually have quick ways to communicate so that players can express ideas and play at the same time.
Specifically, we referred to League of Legends’ (LoL) emote wheel where players could create a wheel of five emotes that could be taken into any LoL match. Borrowing this concept, we imagined users in XR could load their own expressions into a plate that could be taken into any online XR space or game.
We designed the system to have preset emojis to pick from, but we thought the ability to import custom emojis and expressions could allow for more personalized expression.
We came up with three constraints for both preset and imported objects to provide a consistent experience between the two. At this point, we decided to put more emphasis on designing the emojis' visual effects as they are sent to another person.
We compiled a list of interactions that we liked and I prototyped them in Unity for feasibility. If they worked, we would show our class and professor to gauge excitement and then fully prototype the most exciting interactions in Unity.
We made a video prototype to tell the story, which I co-directed and filmed in a commercial-style, that focused on four cases that encompassed our entire design. The other members of my team created and supplied 3D models of emojis in Blender and handed them off to me to build in Unity.
The four use cases we decided on were:
We chose these four because they represented many of the purposes in which someone would use XR emojis. Whether its importing your own objects for self-expression, messing with friends, or throwing a virtual party, we believed these four use cases covered these interactions.
The final design was more magical than we had expected. Other members of our class found it extremely intuitive to grab and throw emojis. Our professor was excited at the potential to throw emojis back and forth with their friends and stated how this felt like the next step for visual communication in XR.
In our early exploration phase, our professor had suggested object alternatives to emojis, as he neither related to emojis nor thought of them as dynamic. Given these concerns, I would like to explore other abstract visual elements as forms of communications in the future.
If we had more time, we would have tried to develop more effects and gestures for our expressions to create more fun ways to activate emoji effects. We currently only utilize the throwing motion to trigger our expressions, but we had floated ideas that involved more dynamic movements.
