For a better experience, we recommend using Firefox and headphones.
Enter

Bringing dinosaurs to life with WebXR

Creating Prehistoric Domain was an incredible journey filled with three years of work, with numerous difficulties, sacrifices, discoveries and success. In the end, a dream came true as we opened a prehistoric park, where visitors can encounter life-sized dinosaurs and other extinct animals.

I am Benjamin Dupuy, the creator and developer of this project, and I invite you to join me on an exploration of its development process. My objective is to provide both an overview and technical details about the development of the project. It is not an exhaustive list, I have chosen to focus on the details that are most important from my perspective.

While I personally coded the entire project, I received great help from collaborators who contributed their expertise in various aspects. They helped procure important assets such as creating 3D models, recording the park's voices, designing the visual identity...

How it started ?

At first, I was merely experimenting with VR. However, I quickly recognized the immense potential, particularly in the realm of VR on the web. Using an impressive dinosaur model, I created a simple VR scene and the potential left me completely astonished.

As a paleo enthusiast and avid Jurassic Park fan, I made the decision to embark on a project with an immersive and educational purpose. With the current technology available, we have the ability to transport people back in time through a virtual park that closely resembles a real one.

The goal is to recreate reality, and with VR, we can engage multiple senses in a convincing manner. Sight is brought to life through virtual environments and high-quality 3D models, while hearing is elevated by spatial audio, enhancing the auditory experience. While hand tracking may not have been implemented yet, the captivating visuals and immersive audio alone can be enough to make people believe in this virtual world.

How Prehistoric Domain was made ?

Web technologies

To make this project a reality, I made the deliberate choice to utilize web technologies, particularly WebXR, a technology that enables immersive virtual and augmented reality experiences directly within web browsers.

There were several reasons behind this decision:

  • WebXR has immense potential, much like web apps have advantages over native apps.
  • No installation is required for WebXR experiences, eliminating the need for users to download and install specific applications.
  • WebXR is compatible with a wide range of devices without the need for extensive custom development. It can be accessed on most desktop devices as well as various VR headsets.
  • By using WebXR, I can harness and combine the power of websites and virtual reality, offering a seamless integration of traditional web elements with immersive experiences.
  • As a passionate web developer, I found the combination of web technologies and virtual reality to be an exciting and fulfilling endeavor.

Basic Architecture

This project consists of several components, including

  • The website
  • The map
  • The various experiences within the park

All developed with a micro frontend architecture approach.

The website serves as the entry point to the park, providing different kinds of information and learning sections. The map allows users to navigate within the park, while the experiences encompass the different exhibits visitors can explore, all of which are compatible with VR.

To achieve this, I implemented a micro frontend architecture. This means that the main website, prehistoricdomain.com, loads separate map and experience components that are developed and deployed independently. This approach offers flexibility and allows me to manage each piece separately.

I can choose the most appropriate tools for each component and make changes only where necessary. For instance, if I want to update the T-Rex scene, I simply redeploy that specific component without affecting the rest of the project.

I used iframes to compose the micro frontends, integrating each component into the main pages. While it may not be the most advanced micro frontend solution, it served my needs well and remained relevant for the project.

Considering time constraints (working on it during my free time) and the need for optimal performance, I aimed to choose lightweight and flexible solutions that included only the essential components required to build the project successfully.

The main website

The main website of the project is built and deployed using Webflow, a powerful no-code tool that I discovered during this project.

Webflow is a tool that closely aligns with the work we do as developers and enables the creation of high-performance pages even without writing code. However, it also allows for the integration of custom code when needed, which was crucial for incorporating the other components and managing specific functionalities such as consent, local storage, and more.

Exploring the realm of no-code development was a personal goal, and after a brief adjustment period, it proved to be a significant time-saver. Webflow provided an efficient solution for creating the main website, and when necessary, I could leverage custom code to extend its capabilities and tailor it to the project's unique requirements.

The map

The map component of the project was developed using Leaflet, and I couldn't be happier with the choice. Leaflet provided a delightful experience, with excellent documentation and a supportive community. It proved to be a lightweight, fast, and user-friendly solution for implementing the map functionality.

To generate the map tiles, I utilized Inkarnate, a DnD fantasy map generator. Initially, I wasn't certain if it would meet my needs, but after some experimentation and adding a few additional steps using Photoshop, I was able to achieve the desired outcome—a captivating adventure-like map for the park.

The combination of Leaflet and Inkarnate allowed me to create an interactive and visually appealing map that perfectly suited the project's theme to enhanced the overall experience for visitors.

The immersive experiences

Prehistoric Domain is built using a combination of Three.js with A-Frame, an excellent tool for creating WebXR projects. The decision to utilize A-Frame was motivated by a desire to build upon a community-driven tool that provides insights into how other developers are building their experiences. Additionally, A-Frame simplifies the process of constructing immersive 3D environments through a declarative HTML syntax.

With A-Frame, I made extensive use of defining components to create shared features such as animations, UI elements, loading mechanisms, as well as independent scenes and related functionalities.

To bundle the experiences, I employed Webpack, generating independent bundles for each scene. This approach allows for flexibility, enabling the creation and updating of individual scenes without impacting the entire project.

The project has minimal dependencies, consisting of a simple JavaScript project bundled with Webpack and incorporating A-Frame and associated packages (many of which are useful, reusable components developed by the community). The objective was to keep the project lightweight and user-friendly. One potential enhancement that could be considered in the future is the integration of TypeScript. This would provide benefits such as improved code organization, readability, type checking, and autocompletion.

Lastly, all microfrontend experiences, including the map, are deployed and served in the Cloud using Hostinger, facilitating the accessibility and availability of the project.

Designing the Experiences

The initial step in the project involved defining the experiences. The first idea was to create a complete car tour of the park. However, this posed significant challenges as it required providing large scene for visitors to explore in every direction, which had a noticeable impact on performance. Despite the trade-offs and challenges, I decided to retain this scene after careful consideration.

For the subsequent experience, I reimagined it as a closed observatory with a large window, reminiscent of what we often see in zoos. This approach allowed visitors to focus on the scene presented in front of them and by the way improved overall performance.

Also by scripting each scene rather than implementing AI, I was able to carefully design and create a coherent and impactful scenario. With this approach I wanted to ensure that the final result felt natural and accurate in portraying the animals.

Timing played a crucial role in the process, as it was essential to ensure that each animation occurred at the right moment, providing a sense of reality and spontaneity. By carefully orchestrating the timing of events, I aimed to create a more immersive and believable experience for visitors.

Concept art played a important role in the project, aiding in the visualization and imagination of each scene, thanks to the awesome work of our collaborator d.s.mokrys_art.

Designing the Website

The design phase of the website played a crucial role, and Elisa Levade made an incredible contribution by creating a the visual identity for the project. The objective was to develop a modern and creative website that evoked the same atmosphere as real animal parks, effectively conveying the feeling one experiences when visiting such a park.

The complete visual identity process was carried out using Adobe Suite, specifically Illustrator and Photoshop. These tools proved instrumental in defining the color palette, creating various assets including icons and logos, and ensuring visual cohesiveness throughout the website. Elisa's expertise helped me a lot for the successful realization of a visually appealing and immersive website that truly captures the essence of the park experience.

Assets Pipeline

In order to delve into the operational aspects of the Prehistoric Domain, it is crucial to address the process of asset importation into the engine. In this section, I will outline the steps undertaken to effectively manage our 3D assets, encompassing their creation, loading, and utilization within the project.

  • The main part of assets comes from Sketchfab
  • I use Blender to apply modifications to the models, it can be
  • Modify and adapt animations
  • Modify materials to fit to the target scene
  • Modify textures
  • The I export the model in GLTF format, 3d format for the web
  • Finally all is compressed and exported into one final GLB file that is load by Aframe

Most of our Dinosaurs models where created by Pxltiger, a very talented 3d artist which help us a lot.

Composing the scene

Scene composition with A-Frame Inspector

The process of creating the complete scene posed certain trade-offs and challenges. Initially, I opted to construct the scene directly in A-Frame, basing it on the different 3D models and components required. This approach provided greater flexibility in manipulating each part of the scene through code. Additionally, loading multiple pieces in parallel was more efficient compared to loading a single large piece. Moreover, I could easily choose which parts to load or exclude based on user-defined performance options.

However, as the scene grew larger, I realized that loading the entire scene within A-Frame could become too heavy and impact performance. To address this, I adapted the process. Given that extensive interaction with the scene was not necessary, I began constructing most of the scene in Blender. This allowed me to export it as a single model, reducing the load on A-Frame. At the same time, specific models required for custom interactions or animations were loaded and added separately to the scene from A-Frame.

By finding the right balance between constructing the scene in Blender and selectively utilizing A-Frame for other components, I achieved a more optimized and efficient scene rendering process within the Prehistoric Domain project.

Audio

Sound design with Premiere Pro

Audio played an important role in achieving immersion within Prehistoric Domain. To accomplish this, I utilized A-Frame's audio features to position audio within the scene. By accurately placing the audio sources, users could experience sounds coming from their exact source locations as they moved within the virtual environment. Additionally, the distance-based volume adjustments make a difference, simulating the natural decrease in sound intensity as the user moved away from the source. These audio techniques significantly contributed to the sense of reality and enhanced the overall immersive experience.

While achieving completely realistic visual scenes may still present challenges, audio has the power to bring us closer to that sense of realism. Often overlooked, audio plays a crucial role in creating a believable virtual world. By carefully managing and utilizing audio elements, we can enhance the sense of presence and engagement, making the virtual experience more immersive and captivating for users within Prehistoric Domain.

Here is the workflow I used to bring audio:

  • Audio sources are downloaded from Artlist or by myself
  • Talented collaborators Dom Bowman and Elisa Levade made invaluable contributions to the Prehistoric Domain project by providing their voices for the park
  • Then I use Premiere Pro to do sound mixing according to a video recording of the scene
  • I export and convert the final audio files into Webm
  • Webm allows for efficient compression and streaming of audio, making it suitable for web-based applications and online content. It helped avoiding gap in audio loop (like footsteps)

Performance

Optimizing performance was crucial for Prehistoric Domain in virtual reality. It aimed to deliver a smooth and comfortable experience, ensuring realism and immersion.

First, I followed as many best practices as possible for A-Frame development. I also utilized useful tools and sought help from the community during the optimization process. Here are some of the most important elements I utilized.

Chrome Dev Tool

Chrome Dev Tools are very important for optimizing any web projects. They provide insights into performance metrics allowing developers to identify and address bottlenecks.

With features for profiling or debugging, Dev Tools provide what I needed to manage ressource usage and focus on performances.

Immersive Web Emulator

Immersive Web Emulator enables users and developers to run and test WebXR content in desktop browsers without using a real XR device.

It's a very solid and convenient way to test and debug the experiences, you can read more here.

Lighthouse

One of the primary tools utilized in the Prehistoric Domain project was Lighthouse. It analyzes web pages, offering valuable insights into performance, accessibility, SEO, and other areas. It provides actionable recommendations to enhance website performance and user experience.

While achieving good scores on Lighthouse is desirable, it is very important to implement the recommended optimizations. The project successfully attained commendable scores in various metrics, except for mobile performance, where Lighthouse simulates slower connections by limiting bandwidth. However, the performance metric was also influenced by the UI choices made, as the project aimed to create unique and creative experiences within the pages to align with its overall theme.

Metrics

I use Aframe stat tool to monitor a lot of thing:

  • fps: frames per second, framerate. Aim for stable 90 fps with the WebVR 1.0 API.
  • requestAnimationFrame (raf): Latency.
  • Textures: number of three.js textures in the scene. A lower count means the scene is using less memory and sending less data to the GPU.
  • Programs: number of GLSL shaders in the scene.
  • Geometries: number of three.js geometries in the scene. A lower count means the scene is using less memory.
  • Vertices: number of vertices in the scene.
  • Faces: number of faces in the scene.
  • Calls: number of draw calls on each frame.
  • Load Time: how long it took for the scene to start rendering, in ms.
  • Entities: number of A-Frame entities.

In addition, I implemented the aframe-fps-counter-component to display the FPS in virtual reality. 

You can see above different sections to consider. The first one is the target FPS, where the aim is to achieve the highest frame rate possible. Monitoring FPS is crucial because it helps ensure a smooth experience and provides insights into the scene's performance. 

When it comes to memory, for example I said early that I initially compose the whole scene into AFrame, it initially led to an increase in the Entities and Geometries indicators, indicating higher memory usage.

The render part refers to the cost of displaying the scene. As the complexity of the scene increases, it can impact the FPS. Despite the challenges, a choice had to be made. In the initial scene, even after numerous optimizations, reaching 30FPS was a challenge.

To render a scene in the web, WebGL technology is used. While it is a solid standard, it does have limitations. New technologies like WebGPU are expected to revolutionize rendering, but until then, compromises must be made. To achieve the goal of displaying high-fidelity scenes, sacrifices were necessary. Instead of using low-polygon models extensively, I opted to push the limits of fidelity by rendering more complex models and shaders. This approach affect the FPS, but it ensures a more realistic experience in VR.

With this decision in mind, lot of optimizations were implemented. Here are some of the optimization techniques that were employed:

Photogrammetry in Blender

Photogrammetry optimisation

Photogrammetry is awesome, it’s a technique used to create 3D models from real-world objects or scenes. By utilizing photogrammetry, you can create highly detailed and realistic models, but it's important to optimize them to maintain performance. So here the goal was to pay attention to reducing the polygon count, simplifying the geometry, and generating lower-resolution texture maps.

Memory & cache management

I defined methods that clears the cache and frees up memory in an A-Frame scene, especially when changing scenes. It disposes of 3D objects, their associated materials, textures, and geometries. This helps manage memory and resource usage, making it useful for scene transitions or optimizing memory in 3D applications.

Texture optimization and compression

Textures consume a significant amount of GPU memory. I optimized each texture by reducing their size employing texture compression formats like JPEG, PNG. Texture compression reduces memory consumption and improves loading times without significantly sacrificing visual quality.

Distance rendering

I used a lot of fog and reduced field of view when needed. Just a small trick to not having to load mush element at the same time, for example during the Tour experience where we move with the car.

Foveated rendering

I usefoveated rendering for every scene. It’s a technique that focuses rendering resources on the area where the user's eyes are fixated, while reducing the level of detail in the peripheral areas of the screen. By mimicking the way human vision works, foveated rendering can significantly reduce the rendering workload, especially in VR applications, resulting in improved performance.

Instancing

Instancing allows multiple instances of the same object to be rendered with a single draw call, reducing the CPU and GPU overhead. Instead of sending individual data for each instance, you can use instance attributes to store per-instance data, such as position or orientation. This technique was particularly useful when rendering many copies of the same object, like trees or grass. I used aframe-instanced-mesh from Diarmid Mackenzie to use that with my AFrame components.

GLB over GLTF

GLB is a binary file format for 3D models that includes both the geometry and textures, packed together in a single file. GLB offers faster loading times compared to GLTF, which is a separate file format that references external assets. By using GLB, you reduce the number of network requests required to load a model, leading to improved performance.

The power of community

The community played a significant role in the development of our project. Throughout the process, I was fortunate to find immense support from various sources. First and foremost, the Prehistoric Domain Discord community grew over time, attracting amazing and passionate individuals who provided invaluable assistance in numerous ways.

Additionally, the broader WebXR community proved to be an invaluable resource, consisting of developers and creators who willingly shared their knowledge and expertise. These individuals are the driving force behind the WebXR ecosystem, enabling projects like ours to thrive. I also decided to sponsor Diego Marcos, the creator of A-Frame. This tool is like the ember of Jurassic Park, the engine that transformed our vision into a tangible reality, making our park come to life.

I am grateful for the collaborative spirit within the community that has made this journey possible.

Other notable points

Water rendering was a big challenge

Water

Water was a significant challenge in my project, it is complex to achieve and has a big impact on performance. To tackle this, I utilized a custom component derived from the Three.js shader available here. One advantage of using A-Frame, which is built on top of Three.js, is that I could directly incorporate this shader and many other custom things when needed. I made adaptations to ensure proper rendering within A-Frame, including adjustments to reflections, movement, and other aspects. The desired effect for the water was to be highly translucent with minimal movement, as it is predominantly used in static water in forest environments, except for the Aviary environment.

Throttle Tick

I started by using tick, a hook function in A-Frame that runs your code every time the frame refreshes, which can be excessive and potentially impact performance. But then I used throttledTick , a function allows you to limit the number of calls and control the update rate, leading to optimized updates, reduced computations, and improved performance.

Delta time

An important concept in animations and largely used in gaming that I used later in the development. I highlight this point because it is a very useful classic.
Basically I used to move animals with custom methods based on curves. It works well but depending on the framerate of the user, the models dosn’t move exactly at the same speed…

In real-time animations, using a deltaTime value allows you to adjust the speed of animations based on the time between frames. By multiplying the animation values with this deltaTime, animations become frame-rate independent. This ensures that animations play at a consistent speed regardless of the device's performance, resulting in smoother and more consistent visual experiences.

In Conclusion

I shared here a part of this incredible adventure, and the journey is not over, there are still many possible ideas and improvements.

Prehistoric Domain was honored with the Education Experience of the Year award at the 2023 WebXR Awards. This recognition is a tremendous reward for all the hard work and dedication put into the project. It serves as a powerful encouragement, affirming the impact and value of Prehistoric Domain's educational experience. I'm grateful for the recognition and motivated to continue pushing boundaries and creating immersive and impactful WebXR experiences.

I hope that you have found the information about the development of Prehistoric Domain insightful and that it has maybe provided you with inspiration to create your own remarkable WebXR experiences !

Feel free to support the project by considering a contribution on Patreon or visiting our official shop The Mystery Dinosaur with plenty of surprises !