PROJECTS
Qiantan Holomap
Date
July 2022
Teammates
Mia Fan
My Role
Ideation
Fabrication
Physical Computing
Virtual Space Design
Tools
AutoCAD
SketchUp
LeapMotion
Unity
OBS Studio
Project Brief
This project is a gesture-controlled 3D Hologram map for NYU Shanghai's new Qiantan Campus. It uses the leap motion sensor as its input, allowing users to use left-hand gestures to navigate through different floor maps at the new campus. Meanwhile, the right hand enables users to choose freely between 12 floors and reset the scene position in case the picture is out of the scene. The work is considered a functional physical installation serving the NYU community, which later will be donated to NYU Shanghai.
Background Research
NYU Shanghai is about to move from the old campus located on Century Avenue to the new one in Qiantan. According to our online survey, students, faculty, and staff are all stressed to get accustomed to the new semester and new surroundings at the same time. Thus, the idea of making an interactive map could help to relieve the tension of getting into a foreign environment. People have already gotten used to the rhetorics of digital media presenting and functioning in daily lives (Bogost 2008). Therefore, a hologram projection could be an easy-accessible option for mirroring a 3D campus map.
A hologram map enables users to have a more intuitive sense of their current position. Unlike the conventional print-out map or GPS navigation map apps, a hologram map would be a detailed portrait of the new campus and every user could redesign the layout by controlling it with gestures. Especially in a university scenario, where a classroom may look the same as another. Since the campus is a 3D space, the hologram map could give a better overview and easier to discover unfamiliar rooms. With the interactive way-finding function, users could have quicker access to the route leading to their destination. Potential additional features such as demonstrating real-time traffic may be implemented as well. Based on current existing hologram projects, we found that the pyramid-like shape is the easiest form in practice. Inspired by Pepper’s Ghost technique that was applied in theater centuries ago, the pyramid shape was proved to be a good way to make a 3D illusion by reflecting four identical images from four directions and ultimately to the center (Gbur 2016).
Citation:
Bogost, Ian. “The Rhetoric of Video Games." The Ecology of Games: Connecting Youth, Games, and Learning. Edited by Katie Salen. The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning. Cambridge, MA: The MIT Press, 2008. 117–140. doi: 10.1162/dmal.9780262693646.117
Gbur, Gregory J. (2016) 'Dircks and Pepper: a Tale of Two Ghosts' Skulls in the Stars website, final page
Traditional 2D Floor Map
Hologram Map
Through our observation study conducted on the use of this installation, we hope to gain a concrete understanding of to what degree, it is influencing and reshaping our daily behavior. We would use other space design cases, for example, Museum4punkt0 in Germany, as models of our recreation of the Qiantan campus in a digitized way. Museum4punkt0 is a project that cooperates with different museum partners and digitalizes the collection with the method of story-telling, AR navigation and/or illustration, and an immersive audio guide. From VR/AR projects to 3D projections, how these new media displays affect our way of understanding “reality” would be invited in the discussion of the establishment of the concept of Metaverse.
Museum4punkt0
In order to achieve what we have proposed above, we have first drawn a flow chart to make it clear what we should do at each step.
Preparation
Original 2D Floor Map
Select Essential Inforamation
AutoCAD
SketchUp (3D model)
Pyramid Structure Construction
60 inch Television
Support and Frame Fabrication (Mainly Wood)
Physical Fabrication
OBS Studio (Duplicate and Rotate for 4 times)
Unity (Shader Writing)
Leap Motion
Virtual Space Design
Preparation
After asking for the original 2D Floor Map from the Public Safety Department of NYU Shanghai, we go through a thorough discussion to determine what kind of elements we should include in the map. If we mark out all the information, it will be too overwhelming and inconvenient for users to navigate through the map. The picture shows our final decision, and this list covers most of the functional facilities and important transportation channels.
The second step is to highlight the elements mentioned above in AutoCAD based on the original map. The simplified version seems more clear and readable to users.
The third step is to input all the sketches we get from AutoCAD to SketchUp so that we can make them 3D models, which later will be used as the base of the hologram map.
Virtual Space Design
After we finished all the 3D models in SketchUp, we bring them into Unity. There is no way people can correlate the picture shown above with the hologram. So we design a customized shader to make it look like something comes out of sci-fi novels.
Then, we tune the parameters of the leap motion to ensure that it is in the right direction and of adequate sensitivity. After that, we do some simple interaction tests in Unity. The following video is to test whether both bands have a rigid body and can interact with Unity game objects.
The third step is to create the UI panel, allowing users to choose freely between 12 floors. The dominant color, since we are creating a hologram map and its sci-fi property, we use pink and sky blue.
The last step is to finalize all the coding stuff, such as how to let hands interact with the UI panel, how the floor appears and disappears according to the user interaction. The primary method we apply is to use the collider and event system.
After several rounds of user tests, this is the most updated version of the instructions, and later, we add one more function to it, which is to reset the image to its original position so that users won't get lost.
Physical Fabrication
Support & Frame
Before we build the final product, we plan to run 2 rounds of prototype tests to ensure the pyramid structure works well for the hologram effect. Moreover, we also want to find out how dark the environment should be so that the image can be seen clearly. At the very end of the round 2 test, we think it's a good idea to use the black curtain to cover 3 sides of the structure. By doing so, the image will be more visible to users.
To make the hologram image large enough for users to see, we eventually decide to use a 60-inch big television as the output. And this will create a 16-inch big 3D image. It's clear that this heavy television cannot stand by itself. So the first step is to prototype the support so that we can know the accurate size of the material we need.
After we figure out the precise size of the material we are going to use, we begin the raw material preparation process. And most of our structure is made out of wood.
The last step is to assemble all the pieces to form a solid structure to hold the weight of the television. After securing the whole structure with the nails, we use black acrylic paint to make it a more balanced outlook.
Pyramid Structure
After the iPad test, we find one issue with the 3mm acrylic board. It will create a double-image effect because 2 sides of the acrylic will respectively reflect the light once and the image will overlap on top of each other. We try to resolve this issue by using the 1mm thick acrylic to take the position of the 3mm ones. But eventually, we find it too hard to support the structure because it will bend due to its weight, which will form distortion on the hologram image. Though we can support it, the effect is not ideal and will make the whole installation look rough.
As a result, we decide to use the 3mm acrylic boards again because it is stronger and can self-stand. The double image is then considered a minor issue because it might be obvious for an iPad-big screen, but it is still 3mm difference for the 60-inch television and in that way the influence can be neglected. Moreover, the leap motion sensor at the front of pyramid will create some space between users and the hologram. This will help to ease the influence of the double-image issue.
Final Result
Assembling all the pieces, we then use the OBS Studio to duplicate and rotate the broadcasting windows for 4 times so that they will be perfectly reflected by the pyramid to form the hologram image.
OBS Studio Demo
User Interaction
We invite different people to try out our project and generate a lot of useful feedbacks before we hold the final public play.
Exhibition
We set the project up near the on-campus cafeteria to hold an exhibition here. Most audience provide us with a positive feedback that the hologram map is easier to navigate through and compared with the traditional 2D floor map, it's a more direct way to know the way to a certain classroom.
At the 2022 Fall DURF Symposium, we share our research conclusion and the project with other faculty members at NYU Shanghai in the form of an exhibition. We have received critical comments and feedback helping us to improve the project.
Reflection
Generally speaking, this is a pretty successful functional installation. During the exhibition, most of the users show strong interest at the first glance of the project and are all willing to try it out. The instructions we made are straightforward and match people's ordinary habits. Based on the feedback, more than 95% of all users reflect an increasing understanding of the structure of the new campus, which originally they might only hear about it on the news. And more than 93% report the hologram map is a more direct way to navigate through the campus relative to their current location.
In terms of improvement, most can be attributed to a more user-friendly UI system. Right now, even if the user clicks the buttons on the panel, there will be no response on the color or any other properties of the button itself, the floor will simply appear and disappear. This makes the process lack "reality", which means that users might be confused about whether the floor actually changes. So adding a feedback sound or simply changing the color of the border of the button will be a huge improvement.