Enriching the Tour experience at CMU with AR
Using new technology to mix reality and the virtual world
Imagine the world from the perspective of 10 years in the future. How much has changed? How much has technology integrated into our lives?
For this project, Susan Ni and I were tasked to imagine that AR headsets have become sleeker, lighter, and commonality in daily life and to think of ways this new technology could aid Carnegie Mellon’s Tour experience. Technology today, like the Microsoft Hololens, demonstrates how this is possible for the future, and new possibilities to integrate technology into an experience is endless. The task at hand was to utilize this technology and design a better way of demonstrating Carnegie Mellon’s strengths to prospective students.
Problem Space
Carnegie Mellon’s current student campus tour model has limitations mainly in the broad difference in range of information student tour guides provide. Since each tour guide has their own perspective and experiences regarding campus life, they are sometimes limited in the specifics of certain subjects, for instance, an ambassador who is a computer science major and has mostly spent time at Gates will not be very familiar with the Design department at CMU.
However, with new technologies such as AR HoloLens, these issues could be solved by developing a personalized and informative tour experience.
Research on Mixed Reality
Expanding the Design process for mixed reality
Key points:
- “Waiting for the developer to finish the prototype” is a common problem for teams
- Quick and fast iteration and design communication that requires little background knowledge helps a team of multidisciplinary individuals work more efficiently like paper prototyping and simple drawings… However, this is limited in the 3D realm.
Methods of quick iteration for 3D
- Bodystorming -> cheap crafting materials to represent objects, digital interfaces and animations. It allows expression and communication of an idea without prior knowledge
- Acting -> perspective of user and see events through time
- Storyboards -> demonstrates the flow of the experience in a 2D medium
Augmented Reality: “The Ultimate Display”
Key pain points:
- We sense an involvement with this physical world which gives us the ability to predictions properties. But we lack corresponding familiarity with concepts not realizable in the physical world
Building familiarity through computers
- Examples of manual-input devices: keyboards, mouse, trackpad
- Other ways which one could interact with a computer include: “drag and drop”, “knobs and joysticks”, push buttons, voice recognition, image registration, gaze tracking
- The computer can easily sense the positions of almost any of our body muscles
- The ultimate display would, of course, be a room within which the computer can control the existence of matter
AR Tour Process
Location
Susan and I decided to choose the Tepper, the business school at CMU, as our location that we would show the AR tour experience in detail.
Tepper is a building that not only caters to business students but is a hub for other students to study, learn, and have fun. It also is the newest building at CMU and features modernism that will be consistent 10 years into the future. Not to mention, the building is architecturally beautiful and gives off a good impression to prospective students.
Exploring Tepper
Upon exploring Tepper, we realized many possibilities for interactions to happen with AR technology. However, to narrow down the options, we created personas to pinpoint goals and what the tour would focus on.
Persona Development
Our personas included:
- A prospective international student, William Huang, who wants to have a good college experience through clubs, food, and social life
- William Huang’s mother, Claire Huang, who is concerned about her son’s well-being and the school’s opportunities for him
- Amy Miller, a prospective student from the Bay area who cares about school culture and building personal relationships with her peers and her instructors
Thinking about Interactions at Tepper
Using our personas, we considered what interactions would be helpful based on their personal needs.
Storyboards
Prototypes
Our prototypes focus on how the user actually interacts with the environment. The tour is going to have a home page available on the wrist of the user that allows them to navigate to the maps page, which allows them to see where they are in relation to the guided tour, or the bookmarked page, which they can save mementos from the tour.
When they enter a location that has a guided tour, they will know from a platform with a cube on it. As they approach the cube will awaken and light up the intended tour path.
The final interaction is when the user approaches the alumni donor wall. The name plates will extend and reveal a 3D hologram of the alumni and they will talk about their experience at CMU. Besides them, a name plate with more information about the them will appear.
Reflection
As the prevalence of digital media in our physical environments increases daily, what is the role and/or responsibility of designers in shaping our environments?
The responsibility of designers is to make sure that the digital media present in the physical environment intuitive and accessible to everyone. Since digital media can be dependent on visual information, it is important that designers keep in mind all types of people and their limitations. For example, someone who can’t read English will not understand written words featured in AR, and as of now, there is no technology that would allow them to translate it. Apps on phones allows people to overcome this obstacle somewhat by scanning the physical environment and translating it; however, in the digital world of AR, where the information not physically there and not detectable by phones, this is not possible. So, keeping in mind varying types of people is required for designers to make well crafted digital media within physical environments.
Feedback and discussion
Our professor, Peter, was enthusiastic about our alumni wall interaction and prompted us to push more. Instead of just a bust of the alumni, how could we utilize the full possibilities of AR and make the experience more engaging? With this, we shifted to a full body representation of the alumni and started changing the prototype.
AR research
Companies building AR or VR headsets
Google —Google Cardboard which allows a more affordable option to VR, replaced Google glass which may be brought back for industrial use
Microsoft— HoloLens for AR and there was a release for the HoloLens for industrial application as well
Apple — ARKit a project in development as well as N421 and N301 VR headsets
Valve — Index for consumer VR products. Well integrated into Steam
Magic Leap — Inspired by science fiction novels and is developing for target defense, healthcare, and industrial manufacturing
Video sketches: Exploring Pervasive Computing Interaction Designs
Using video sketches to demonstrate computing concepts reduces cost and production issues as well as allows for students to create a product that allows viewers to envision how someone would interact with a product.
Interaction Prototyping 2.0
Considering the value of each interaction we brainstormed and the limitations of time, we decided to cut down our tour to three main interactions: Alumni donor wall, season changing, and classroom demo.
Alumni Donor Wall Interaction
To display our alumni, I thought it would be fitting to have them stand on a podium and talk about their experience. This would give some grandeur to the experience and present the alumni as the accomplished people they are.
Animating and creating the model was a fun experience! There were a few difficulties that limited the design (shade smooth on cut out circular objects caused bugs), but overall I was happy with the podium.
This interaction focuses on featuring the alumni fully through AR. Once the visitor presses on a name of an alumni, a podium will appear with that alumni projected onto it in 3D space. A panel will appear with more information about the alumni such as name, graduating year, major, and where they are now. They will then retell a short paragraph about their time at CMU, which will give the visitor more insight as to why they should choose Carnegie Mellon.
Menu Bar — Susan
The visitor will have the menu bar visible in the left side of their hololens. This will allow them to select the camera, share button, map, or bookmark button. Having this addition also allows the viewer of the video to see clearly what is 3rd person and what shot is 1st person.
Guided path — Susan
As the visitor selects a certain path, these indicators will appear on the floor. Once they get closer to it, they will spin and then disappear. This helps the visitor have a general idea of where they should go without requiring a lot of attention. Other ideas for this was an arrow that navigated the visitor like a compass; however, this would need too much cognitive consciousness to see if the arrow changes and would prevent them from looking around at the campus.
Classroom Visit
This interaction allows for the visitor to see classes in session. Typically, the tours at CMU show an empty classroom, which is difficult for the prospective students to envision their experience actually being in a class. This AR addition allows for them to watch classes of their choosing with real students and professors without disrupting a class.
The interaction involves a panel of classes of which the visitor can swipe through. Each class has a description and a video that can be played of the class session. Once the video is played, the visitor can move around in 3D space and view the students and professor.
Feedback and discussion
- Time changing interaction is pretty but what does it showcase that other universities can’t?
- Perhaps change the blue to a less saturated blue
- Build the tour on one the alumni interaction rather than have them disjointed, so it will flow more like a cohesive story.
- Alterations to details like opacity of screens and alignment
Alterations
After the feedback, we shifted to a narrative centering around the alumni. When the visitors enter Tepper, they will be greeted by an alumni panel that allows them to choose which alumni’s story they want to hear about. We have incorporated three paths: American undergraduate major, master student alumni, and international undergraduate student. The alumni will then talk about their experience at CMU while taking prospective students around Tepper. One of the main stops will be the alumni showing their favorite classes.
Interaction Prototypes 3.0
Alumni Wall
Altered to start with an alumni tour guide selection (undergraduate, undergraduate international student, and masters student). They will then be taken to the alumni wall to learn more about their specific alumni. The alumni will talk about their experience at CMU and then bring them to the classroom viewing interaction.
Menu Bar
This menu will be present at all times if the visitor looks down at their arm. The main circle has a map that lets visitors see where they are. There are also features like sharing to social media, taking pictures, and bookmarks.
Guided path
Classroom Visit
Altered from tinted blue to show more of a realistic version of the classroom session and to provide better visibility.
Snap Shot
Visitors must hold up their hands to take a photo and a count down will start. After the photo has been taken, it will save to bookmarks and will be uploaded to the cloud for viewing at home.
Final Video
Reflection
How were the skills you developed in the first project similar and/or different from the second project? What is your understanding of the role of an Environments designer?
This project involved more digital environments compared to the first project. Since the visitor sees the world in a new lens, one with AR projections, the design of the interactions can be boundless in terms of feasibility. Moving towards a more digital world creates a dynamic that is both unlimited and limited at the same time. Although almost anything is possible in AR—a person can appear out of thin air or words can dance around a visitor in 3D space—there are new and unfamiliar boundaries. Experiencing AR in real life allowed for these boundaries to establish themselves better in my mind, for example, how some interactions may feel better in the physical world like pressing a button that has dimensions and that invokes the sense of touch or typing on a keyboard with the familiar click-clack of keys. The interactions that we designed have to make sense in the digital realm and coexist with the physical realm. My understanding of the role of an environments designer has shifted to include more thoughtfulness to the human senses. Although most of these designed experiences rely on sight, environments designers must think about what really makes an experience fuller and incorporate all the elements—whether that be sound, touch, or smell—into their designs.
