The first Deus Ex game from Eidos-Montréal used Crystal Dynamics’ CDC engine, which did not meet the needs for Mankind Divided. The team soon decided to go for IO Interactive’s Glacier 2 engine, but ended up heavily modifying it, resulting in the creation of its very own engine: Dawn Engine.
When Deus Ex: Human Revolution was released in August 2011, we were all relieved to have been able to see this through. Since we started working on the project, we had heard all the stories about the franchise. Eidos-Montréal was created in early 2007, and a core team would be assembled in May of that year to work on the mythical Deus Ex franchise. When I think about it, it was quite a daunting task. Not only did we have to create a new studio and draft as much talent as we could, but we also had to begin work on the third installment of a beloved franchise from the early 2000’s. Indeed, we had heard many stories about the franchise. While our studio was new, we worked closely with multiple people at Eidos Interactive’s head office in Wimbledon, Crystal Dynamics in California, and IO Interactive in Copenhagen, who had been witness to multiple attempts to create a sequel to Invisible War without success. The pressure was definitely on.
From CDC for Deus Ex: Human Revolution …
When it comes to tech, we set our sights on the Crystal Dynamics’ engine (CDC). We were quite impressed by the history of the engine and its overall performance. We had quite a lot to accomplish, and a lot of content to fit in just one game. It was clearly the best decision at the time. We knew little about our game at the beginning, but discovered new things along the way. Games had evolved quite a lot since 2000, and we knew we needed to refresh the franchise while still keeping its core intact. So, we ended up with a long list of things we needed to do to adapt CDC for the purpose of creating a new Deus Ex, and new items would be added to the list as we refined the idea of what Human Revolution would eventually become.
We collaborated with Crystal on the development of these mutually beneficial changes to CDC for a good chunk of time. We eventually moved on to an isolated branch, where we pursued more drastic changes that would make DXHR possible, but were not desired for the Tomb Raider franchise. We clearly struggled on the dual mandate of creating a game that we had not fully defined yet, and also adapting the engine to this ever evolving idea of what Human Revolution would eventually be. One of our major tech undertakings was related to the Detroit level. It is what we call a city hub: a huge, open ended level that the player would revisit several times in the course of one walkthrough of the game. Not only would it contain elements driven by the core story, but the player would be able to roam the city and discover new things to do, including new side-quests. It required adapting the streaming system, creating a host of new tools to cope with multiple layers of gameplay, adapting the shape of the city depending on the point in time the story takes place, etc. The amount of content we crammed in that level was pushing the CDC tools to their absolute limit. People who had to work in the Detroit level suffered quite a lot. It would take an increasing amount of time to build assets to see your work in the game, the editor performance would degrade over time, etc. Obviously, a lot of the programming effort was spent to alleviate the pain related to the pipeline, but it was still a long and arduous process. Even as the main team was busy closing the game, we started looking at ways to improve our lives for the sequel.
… to Glacier 2 for the Sequel of Deus Ex: Human Revolution …
A few months before the release of DXHR, a small team of programmers were indeed looking into some major changes we could make, with the goal being to help fit our vision of CDC. We wanted to make sure that, this time around, the engine would be able to cope with the amount of content required by a Deus Ex game. Those changes included paradigm shifts on how to build raw assets for the target platforms (Xbox 360 and PS3 at the time), ensuring the editor was able to better show what final assets would look like when built, etc. Those changes were too ambitious and risky to push during an ongoing production, but we wanted to have a head start for the sequel. When DXHR was released, we mandated part of the core programming team of DXHR to undergo a large scale engine evaluation, with the goal being to investigate both commercial game engines and internal ones. The idea was to leave no stone unturned. That may sound a bit drastic, as continuing with CDC was the safe choice, especially considering we had proven it was quite capable of supporting the production of a Deus Ex game. On the other hand, it had been 5 years since we started to work on the game, and it was clearly a good time to re-evaluate the tech for our next move. Given the Thief team was using Unreal 3, it was obviously on the top of our list of commercial engines to consider. We also evaluated Unreal 4, which was in its infancy at the time. As for internal technology, we looked at the improved version of CDC that Crystal had been working on for Tomb Raider (2013) and IO Interactive’s Glacier 2, which was a complete rewrite they were developing primarily for Hitman: Absolution.
»We were not only looking at selecting an engine for the next Deus Ex, but also for any games that would follow.«
That small team of 3 programmers found out Glacier 2 was addressing a lot of the issues we were having on our local version of CDC. On the other hand, Crystal had also been making extensive changes to CDC for the next Tomb Raider, and we had also made our own set of improvements. Both engines were very strong contenders for our next ambitious take on Deus Ex. The fact that the renderer in the Glacier 2 engine was closer to what we needed, and the toolsets were addressing many of the issues we had just gone through for DXHR, were enough to make us strongly lean towards selecting it for the next big installment in the franchise. A lot of the new concepts we had to introduce in CDC for Human Revolution were already there in Glacier 2. So that was already something we could safely assume would work right out of the box for DX4 (which was the codename of the sequel to Human Revolution at that time). Our next game would, however, ship on the next wave of consoles. That meant we needed, at the very least, to convert the engine to 64 bits, and the renderer to DirectX 11. Early 2012, we made the choice to officially select Glacier 2 for the next Deus Ex.
That said, making that decision was not easy. One has to keep in mind that we were not only looking at selecting an engine for the next Deus Ex, but also for any games that would follow. That was actually a big part of the conclusion of the technology evaluation, since it did relate to how we were seeing game development and workflows for the future of the studio. The next big challenge was to port the »game layer« of Human Revolution from CDC to Glacier 2.
… to Building The Dawn Engine
Moving some high level systems from one engine to the other (e.g. much of the AI subsystems) was a viable option, as it only had limited dependencies on the core subsystems of either engine. For other systems though, no direct conversion was possible. Several gameplay systems were relying on the CDC scripting engine. However, Glacier 2 does not have a text-based scripting engine, instead opting for a very powerful »Entity System«. One can visually assemble behavior blocks to create more complex behavior. The whole engine is built in a way so that many, if not all, of the core subsystems can be interacted with through the Entity System. There is not much you can’t do with it. The obvious drawback is that all of our scripted systems needed to be rethought and completely re-implemented when moving from CDC to Glacier 2. We collaborated with IO on multiple topics in the first year of our adventure with Glacier 2. They provided a lot of support, which ensured that our content creators and programmers were familiar with the philosophy behind Glacier 2. Both studios also actively collaborated on several improvements to Glacier 2. For instance, we worked together as a single team to replace the animation layer, which had been in place since the engine had been originally conceived.
Obviously, we added a lot of requirements of our own. We did a lot of additional work on the rendering side, and a lot of care was put in setting up a Physically Based Renderer. We invested in Global Illumination techniques, and offline processes for pre-computing light probes used for the second bounce, etc. We also reworked the streaming system extensively, serving the purpose of supporting the open-endedness of our new city hub (Prague), as well as that of the other levels with multi-path, back and forth, etc. These were new constraints that did not exist for Hitman: Absolution, but were very clear for the Deus Ex franchise. One of the other big additions we had to make to the engine was to support shape shifting levels, much like we did for Human Revolution. Prague, which the player will visit multiple times during their play through, was intended to have both a day and a night version, changing weather conditions, and would even behave differently depending on when the player went there. City blocks would be barred or opened depending on when you visited, and population, along with overall AI behavior, would change based on the same criteria. Even if you visit the same location twice, it would be much different.
On the tools side, we had to add support for Deus Ex specific pipelines, including our conversation and character systems. When we added these tools, we developed them with WPF, which signified a move away from WinForms, the previously used framework in the original Glacier 2 engine. Moving to WPF allowed for increased performance for UI intensive workflows, as well as cleaner architecture, to clearly separate the UI work from the algorithms they leveraged. Exactly five years after the initial release of Deus Ex: Human Revolution, we released its sequel Deus Ex: Mankind Divided. While we are pretty happy with how the game turned out overall, there are obviously some key areas where we’re looking to improve the Dawn Engine.
Since the studio started back in 2007, we have made forward looking strategic investments, focused on the future of technology. We now have a dedicated team for Dawn Engine, which is taking care of longer term improvements to its tools and pipeline, as well as the runtime of the engine. The idea is to ensure they are not caught in the immediacy of game production, and can make technology related changes geared towards the requirements for the next 6 to 12 months, while still accounting for the needs of the games currently in development. They will therefore tackle more complex, or more risky, tasks. We also created an R&D department, called LABS, almost 3 years ago. While being labeled as an R&D team, LABS is still very closely involved with production. We are not simply talking about research topics that will only see the light of day in 5 years. In fact, LABS has already contributed to 3 games since we assembled the team (Rise of the Tomb Raider, Hitman and Deus Ex: Mankind Divided), and worked on various technologies, such as hair rendering and simulation, snow rendering, ocean rendering, volumetric lights, Screen Space Ambient Occlusion, new lighting or occlusion techniques, and loads of kick-ass VFX.
Both these teams started working on the future of Dawn Engine long before Mankind Divided actually shipped. Even though the DXMD team was busy getting the game ready for release, we already had a long list of things we wanted to investigate for the next wave of games. Dawn Engine (and the original Glacier 2 for that matter) being what it is, doing simple things can be quite complex, and the learning curve is often quite steep. For instance, the version Mankind Divided shipped with had very basic tools for particles. As stated previously, when you want to make a new particle system, you need to do it by assembling a few entities to get you going. Doing anything somewhat fancy can very rapidly become quite technical and tricky. Recently, we added a proper particle pipeline that greatly simplified things. Creating and configuring even complex particle systems is now not only simple, but consequently reduces iteration times by a considerable amount. Ultimately, assets created in this manner are of a much higher quality, as more time is spent working on the art, and much less on fighting with the tools. At this point, this is the strategy we are going for in pretty much all areas of content creation. On top of this, we have added support for GPU particles, while also reworking the general architecture of the particle system. All in all, particle simulation runtime performance has already improved by 30 percent. Recently, we also made huge modifications to the physics engine, which also yielded a 30 percent to 50 percent improvement over the code that shipped this year. We are hopeful this will give our designers much more flexibility when it comes to adding life to the levels, or coming up with neat gameplay ideas that leverage physics.
Another big topic we are keen on investing is animation. While we have been developing a new animation system in Dawn Engine since we started the working on Mankind Divided, there is still plenty left to do. Our current implementation is basically an animation graph that animators can fully control. It does what it has to do fairly well, but still requires a cumbersome setup to get things in motion. Again, these powerful tools give you plenty of freedom to shoot yourself in the foot, so we are looking at new ways to not only simplify the tools, but simplify the animation system as a whole. We have a few ongoing R&D topics focused on producing significantly higher quality body animations with less effort. I can’t really get into the specifics at this stage as this is very early work, but hopefully we will be able to talk more about this in the not so distant future. We are also specifically looking into facial animations. We know we have a huge margin for improvement, and also have some really interesting ideas on how to make them much more realistic. We are basically overhauling the whole pipeline from motion capture up to the runtime.
There is still a lot to do and, as a matter of fact, we are still learning a lot from Mankind Divided. Since the release of the game, we’ve been continuing to deliver patches for it. Of course, we are fixing bugs that unfortunately made it to the gold master, but we are also extending the Breach online mode, provided alongside the story mode. We are releasing content every month, and monitoring how our players are interacting with the game, not only through community forums, but also through metrics collected by our servers. We are literally going through gigabytes of data to understand how the game performs, along with what does and doesn’t work.
The wealth of data we have gathered in the weeks following the release has proven invaluable, not only on the creative side, but also in helping to make technology related decisions for our next projects. An example of this is how we will adapt our multiplayer approach in the future: We are already doing the groundwork to support a newer vision for multiplayer and, this way, ensure that the technology will not be a limiting factor, particularly when trying to bring new and innovative ideas to life. Of course, this applies to several other areas of the engine as well.
Overall, I am quite happy with how the game turned out, and the solid game engine we now have. There is always going to be something that needs improvement, but that’s who we are. That’s really what’s keeping us motivated every day.
About the author:
is Chief Technology Officer
Julien started his career as Programmer at Ubisoft in 1998, working on projects like Splinter Cell: Double Agent, Splinter Cell: Chaos Theory, and Rainbow Six: Raven Shield, as well as two other unannounced games. In 2007, he joined Eidos-Montréal as Lead Programmer, becoming Director of Technology in 2008, and Chief Technology Officer in 2015.