Overview
My overall immersive design project is designed to be more of an experience than a narrative-driven narrative. I have used some more niche parts of the Alice in Wonderland concepts, such as just a representation of the Madhatter village having houses at the start behind the viewer, giving them something to look at. This is to help correlate to the resemblance that The Mad Hatter and Cheshire the cat have, as they can often be found together. I have used a custom particle effect of blue firefly-like particles to represent Cheshire. My final video render will be at the bottom of the page, above my references.
Research and considerations
Within virtual reality-based projects, there are some big considerations to keep in mind, both in design and the consumption of the end product. For myself, my biggest consideration is what is going on within the scene. As it’s a 360-degree space, all views and angles can be seen at any given time, so I need to keep in mind what is in front, behind, left, and right. Not so much up as for my scene, which will be an open, dark sky fitting my nature/camping theme.
Key things to keep in mind for virtual reality overall, however, are:
Sensory Immersion: As I previously said, this is my biggest consideration: what is going on in all views, and can the scene portray the rest of the senses effectively for its setting? Such as sight, sound, touch, and smell. While it’s only a 3D render, touch and smell are out of the picture; however, it’s the imagery that matters, letting the consumer of the product get a feel for what they could experience.
Emotional Immersion: Does the scene have a narrative, and how could this narrative impact the viewer? For me, this won’t be so much of a focus, but with it being a nature-based sequence, could it help the person feel at peace and calmed, and how could this be achieved? On the other hand, if it were something like a game or movie, how could dialogue or actions impact the viewer?
Interactive Immersion: Does the VR project have interaction or allow for a sense of touch? Not all virtual reality projects have to be interactive, especially if they are video-based or designed for immersive experiences. If it is a game-based project, for example, does the viewer or consumer have a sense of control over what is happening or be able to manipulate what is going on in their surroundings?
Narrative Immersion: Not just games, but books and movies have narratives. Does a virtual reality project or experience have a narrative as well? If so, it should allow the consumer to feel like they are within the environment and the setting in which the project is being portrayed; my project would have to bring a sense of wildlife and nature aspects to it.
storyboard and spheres
For my story boards, I have chosen to do something I have never done before: use both story boards and story spheres, which I had never used before. Along with this, I have attempted to create each scene in front and back view for what the viewer should be able to see, keeping in mind that the project is designed for a 360-degree field of view. Along with this, I have chosen to capture the six key points I intend to capture within the full sequence. I won’t be including the duration of each scene in the storyboard description as I am undecided about how much time I feel the viewer would require in a 360-degree space to understand what is going on.
Scene 1: Where the caera starts and can see down a road towards the campsite along with some housing behind it to represent the mad hatter’s village in a large way to help tie it together with a planned visual effect
Sound: ambient nature sounds, crickets, etc.
Scene 2: The camera approaches the campsite entrance, where it can see mostly fences in front of it; however, it can pan around to see the rear view where it came from, being surrounded by a forest, as well as being able to peek at the campsite, which I intend on blocking most of with a rock until they get there.
Sound: Ambient nature and subtle fire crackling as they approach
Scene 3: The camera approaches a water well as the firefly VFX draws the camera towards it. The well is public and is fenced in with a map of the campsite potentially.
Sound: Ambient nature and quiet running water as the stream is near.
Scene 4: The camera is now within the first campsite experience, where there will be several tents along with a crackling fire for light. The ground will be littered with food items or other personal belongings someone may take camping.
Sound: crackling fire and ambient nature
Scene 5: Scene 5 brings the camera and viewer up to a bridge, which will lead the scene into a new campsite. I did this rather than a whole new scene, as the theme was very much the same, just with a slightly different environmental setting.
Sound: Running water and an alternative nature track
Scene 6: My final scene, for which I have a storyboard, would be the ending part of the scene, which will last just as long as the start, as I have a firefly effect I will make, and it will guide itself out of the scene, going under the bridge and up the brook.
Sound: running water, fire, and an alternative nature track
Aditional concept plan
I considered trying to make the experience more interactive, like a game, but I could not manage to do it using a 360-degree camera. The idea was to use white teleport pads that would be triggered by a gaze interaction, where the viewer would look at the teleport pad, and it would have a 1-second timer while the viewer looked at them before being teleported. It would retain the viewer’s current camera position to not give a motion sickness effect or disorientation as to where they are looking at that given time.
Along with the teleport pads, I had a concept for interactable objects that would start spinning once activated. These could be used to signify effects applied, such as my original idea for shrinking or growing (eat me, drink me) effects from Alice in Wonderland, as well as time changes and other Alice in Wonderland-based effects.
Story Structure
Using Unreal Engine 4, I have used the landscaping tool along with a combination of manually placing foliage and the foliage paint tool to help decorate the environment, giving it a bit of a rustic wild life look while trying to keep the concept of Alice in Wonderland in mind.
As I said in my overview, my immersive design is not so much a VR game project as a narrative-driven project. I am attempting to create a form of immersive campsite within Unreal Engine 4 using the OffWorld Live 360 Camera (OffWorldLive, 2021) as a way of experiencing what a campsite could potentially feel and seem like for someone who has never experienced it before. Using assets found free on the Unreal Marketplace, I have managed to create a form of campsite using tents, campfires, signs, and a lot more assets within their respective packs, which are referenced at the bottom of this page. My approach to Alice in Wonderland is more vague and niche in the way I am approaching it. The rabbit hole is very vague and more open-minded depending on people’s thoughts, views, and imaginations of what campsites might look like. This vague approach isn’t my only Alice in Wonderland consideration, as I have a reference to The Mad Hatter in the terms of a few houses put together to represent his original village, but it helps tie the emphasis to what I aim to do for Cheshire the Cat, guiding the viewer through the campsite and drawing attention to key parts.
Digital affordance
Firstly, looking into digital affordance, what exactly is it?
Looking at an article on ResearchGate (Nöll, 2021), it appears to be the correlation between objects and the user experience. This could be effects that draw the attention of the viewer or things such as audio placement being directional or specific to certain locations. With this in mind, I would start considering how to approach this within my own project, especially with it being a nighttime scene at a campsite. I previously intended to do two different scenes, daytime and nighttime, using the interactive objects I had prototyped. For my digital affordance to work effectively, I have chosen to only do the nighttime sequence as it best shows the fireflies and Cheshire cat guide around the campsite as they stand out more at night as well as having a faint light emitter applied to them for better visibility.
As I wanted to keep my scene quite minimal and not overdo the visual effects in consideration of virtual reality motion sickness, I approached my project as a more visual experience of what a nature-based campsite could be like to someone who had only been in log cabins or had never been to one at all. With digital affordance in mind, my plan is to use directional audio cues for things like running water and campfires, which also symbolise the importance and life of a campsite, as well as audio for wooden bridge walking. I would go on to have issues with this, which I will explain in the audio section. Besides audio, I am using my blue fireflies in a reference to Cheshire the Cat from Alice in Wonderland to help draw the viewers attention to certain details of the scene as well as draw the camera in those directions while moving. My camera itself will be moving throughout the scene; however, I have kept the speed of this moving slow to further avoid motion sickness, as there will be no harsh transitions or fast moving required for my concept.
Audio
With audio, I have had several complications in my approach to how to implement it. Due to personal issues, I’ve been unable to attend lab sessions, and I am unsure how to use DearVR for audio. My original idea was to have audio tracks applied to objects within Unreal Engine 4 to give proper directional-based and immersive audio depending on where the viewer is looking or the camera is placed at any given time. This approach would give the best audio directional placement as well as immersion, but it wouldn’t be without its issues, as I would find out during this. My audio would be implemented in Adobe Premier Pro using free assets I had found, which are referenced at the bottom of my dev log along with the assets I found. Given my time constraint and deadline, I had to do the best I could with my time, so I opted to just use a stereo audio track for all of my project without localization or directional audio, much to my dislike.
The image with the campfire has two orange circles around it: an inner attenuation radius and an outer one. The further away from the source of the fire, the less audible the source of the sound will be. In concept, this would work perfectly; however, due to using the 360 Camera plugin, I am unable to record the audio source with the rest of the scene. This has caused major issues with the way I know how to implement audio within my immersive experience. As I do not know how to use DearVR, I am going to have to implement a less experienced and knowledgeable approach with implementing audio, which may not cause the desired effect I would have wanted.
Embodiment
For virtual reality, there are several approaches to immersion, including embodiment. Embodiment means that the viewer has a physical appearance in the virtual reality space. As I am attempting a campsite experience, I am not looking to include a body, including no arms and no legs. My sequence is primarily an immersive look into what a campsite could be to someone who has never been; this is part of why I decided not to include the body, as it would not be a necessary inclusion, especially without physical interactions between the viewer and the environment.
Development
Before I started designing, I had to consider how I would actually render out the scene within a 360-degree space, as the Unreal Engine doesn’t have the best native support for this. Through looking online as well as talking to lecturer Jason, I decided to use the OffWorld Live 360 (OffWorldLive, 2021) camera plugin for Unreal Engine 4. It does support 5, but I opted for 4 as it’s what I’m most comfortable with. During the development of my scene, it was all done within Unreal Engine 4, as previously stated. Starting off with a flat plane of the landscape tool, I started setting out the basics of the first main campsite, which I have designed to be more communal with several tents and a shared fenced-in area. I opted to keep the tents to five just to make sure there wasn’t an overwhelming amount going on in the scene for my project. I also started toying with the idea of having an entrance as well as a public well for fresh, clean water. The back of the tents leads off into some other environmental area that I opted not to use for the VR project to keep the video and experience within a good time frame without being overwhelming.
Between the brook implementation and foliage, I started to litter the camp area with some debris and objects that would commonly be found among people potentially going camping, such as BBQ roll-up sleeping bags and bits of food debris that might have been dropped or left there by people who are staying in this camp. Alongside this, I was starting to implement foliage more within the scene. I included some logs around the fences that the groundskeepers would have potentially put there as some more decorative aspects in the main camp area, along with stone tiles on the ground.
Unfortunately, after here, I had issues with documentation and don’t have much to show between here and my final product. However, between now and the final product, the main changes I had made were the implementation of my fireflies to guide the viewers attention, keeping in mind the Alice in Wonderland theme, but I also added some houses for the mad hatter, built a bridge to help transition between the two scenes, and also changed the scene from daytime to night as I felt it best suited the campsite experience with the audio I had chosen alongside my fireflies.
As I started fleshing out the environment some more, I enabled the water plugin, which is installed with the Unreal Engine but is disabled by default. Doing this allowed me to make a natural-looking murky-ish water stream, giving it some more immersion and depth in the environment and helping tie together my campsite experience. Originally, the water colour was a bright, vibrant blue to represent what I assume is more of an ocean or clean pond; however, due to the design of my environment, I felt a more pond-like, greenish water would be more fitting, as there would be dirt and algae growing in the water.
I used a combination of hand-placed foliage as well as the foliage tool within Unreal to give a more naturalistic placement to the main forest density as well as the grass. I did not go over the top with the grass placement to help keep render times down as well as the overall design of the environment. I had gone with a semi-realistic approach to the design, and I felt like overdoing it would cross that boundary to more full realism and could be overwhelming when in a virtual reality environment. Along with this, I designed a more privateized campsite that might be viewed if people pitched up tents near a river; more like a “more natural” style of camping going out into nature.
To create the 360 video, I rendered out every single frame within Unreal Engine 4 using the 360 camera plugin from OWL (OffWorldLive, 2021) and then imported them into Adobe Premier so I could add some audio and also a good-looking but short blend sequence between campsite 1 and the start of the next scene. Once I had a result I was happy with after a painstaking amount of experimentation and failed attempts, I imported it into Adobe Media Encoder, where I could easily add the “virtual reality” aspect to it, making it into a 360-degree video for my submission. I did not use any special filtering or post-processing to help fit the natural environment theme I had created.
Varierty of 360 render tests. Numbered for posting, not in order.
Reflection
Overall, I am very happy with the outcome of my immersive design within a campsite experience. I have had a fair few issues along the way, such as figuring out this new plugin for the 360 camera, as the official documentation was incredibly rushed and felt very uninformative. However, once I had tackled this, I felt a lot more confident in what I was doing. One big thing I do not regret was not using the Viper render farm and using my own hardware for this project. The longest render I had was when I overshot the resolution required and did 60 fps; it took a total of 2 hours. This was a good learning experience, but it also felt important to me for my future goals and projects after university. The composition of the scene itself I am happy with, but I wish I had done a bit more and made it larger and longer. I was conscious of time constraints and getting a finalised product for the assignment. My biggest regret is audio. I had some issues with it and could not localise audio to specific parts of the scene within my timeframe. I chose to go for a very vague Alice in Wonderland kind of approach, using more niche parts of the universe that I felt might not be utilised outside of the movies. This was with my blue fireflies representing Cheshire Cat. My only concern with this was the lack of depth until I had the rest of the scene in place. Overall, I feel like my environmental design skills have come a long way since I first started doing it in my second year, and I have learned a lot of useful skills within the Unreal Engine that I will be implementing in future projects of my own. It has felt like a very rewarding experience. Near the end, I opted not to do a separate demonstration video for this assignment due to the time constraint; however, I feel like my development log should have enough explanation in its final form.
Final Product
By default adobe media encoder has the video zoomed in a small amount. The scale of the zoom does not affect the final product, if it looks blurry, zoom out. Any fuzzyness will be down to youtube bitrate compression, Had it in media encoder for 100k-120k bitrate
4k Resolution 60 FPS upload.
Refrences
Advanced Asset Packs (2015) Advanced Village Pack in Environments – UE Marketplace. Unreal Engine. Available online: https://www.unrealengine.com/marketplace/en-US/product/advanced-village-pack.
Epic Games (2020) Edith Finch: Sam Room in Environments – UE Marketplace. Unreal Engine. Available online: https://www.unrealengine.com/marketplace/en-US/product/ef-sam.
eric5335 (2008) meadow ambience.WAV by eric5335. Freesound. Available online: https://freesound.org/people/eric5335/sounds/53380/.
Glaneur de sons (2007) Little fire.ogg by Glaneur de sons. Freesound. Available online: https://freesound.org/people/Glaneur%20de%20sons/sounds/29727/.
InspectorJ (2016) Stream, Water, C.wav by InspectorJ. Freesound. Available online: https://freesound.org/people/InspectorJ/sounds/339324/.
mrrap4food (2022) Small camp fire.mp3 by mrrap4food. Freesound. Available online: https://freesound.org/people/mrrap4food/sounds/619040/.
NatureManufacture (2017) Environment Set in Environments – UE Marketplace. Unreal Engine. Available online: https://www.unrealengine.com/marketplace/en-US/product/environment-set.
Nöll, R. (2021) Can’t touch this? – Exploring the Concept of Affordance in Virtual Reality. ResearchGate. Available online: https://www.researchgate.net/publication/353351476_Can’t_touch_this_-_Exploring_the_Concept_of_Affordance_in_Virtual_Reality.
OffWorldLive (2021) OWL Live-Streaming Toolkit. Off World Live Limited. Available online: https://offworld.live/products/unreal-engine-live-streaming-toolkit.
Ruben_Villar (2022) Berrea 20220919.wav by Ruben_Villar. Freesound. Available online: https://freesound.org/people/Ruben_Villar/sounds/654100/.