By João Vítor Silva

Intro

Hi everyone! I’m João Vítor Silva, a Master of Science student in Computer Science at the State University of Campinas, Brazil. Coming from a Computer Graphics and Machine Learning background, I’ve been involved in NVIDIA Ray Tracing and real-time rendering research projects for the past three years. In January and February 2020, I worked with DGG, the creators of RapidCompact, as a Graphics Programmer implementing a prototype of an Autodesk 3ds Max 2020 plugin. The company’s CTO, Miguel Portela Sousa and their 3D Solutions Engineer, Julian Neagu supervised my work. This blog post will be a recollection of what we learned during that time, and where we can go from there.

The General Problem: Preparing CGI Models for Real-Time Rendering

RapidCompact, as well as DGG itself, aims to make interactive 3D graphics accessible by anyone, from anywhere. One common problem they identified is that e-commerce companies, for example, furniture retailers had high-quality CGI ready models with complex materials and wanted to make it available on a product page for end-customers to view and interact. There was, more importantly, no automatic way to convert V-Ray based materials from a 3ds Max model to real-time materials to be used with an FBX model – this had to be done manually for each mesh, material and model which could be very time-consuming.

Project

Wanting to streamline this process, DGG contacted me to work on a proof-of-concept. Ideally, we would come up with a plugin to bridge the gap between 3ds Max and their own software, RapidCompact, by approximating, converting and exporting the V–Ray materials as textures for FBX files. This plugin would access an asset’s materials through the 3ds Max software itself, map existing V–Ray materials to PBR materials and export them. Down the line, the plugin would also make an integration with RapidCompact through the use of its own API, allowing it to optimize the model.

Challenges

The project started in January 2020 and from the get–go, presented itself as both a great learning opportunity and a challenge. While I had previous working experience with C++, the 3ds Max API and even the software itself were still unknown to me. Several other concepts related to the materials themselves and how they are used in practice were also new. Miguel and Julian supervised the whole process helping me whenever I had issues.

After testing and investigating how the 3ds Max plugin API worked, we concluded that the best approach would be a hybrid plugin, also making use of the Autodesk MAXScript language, managing to make the most of the software. The 3ds Max C++ plugin API has several strengths and performs better as its code compiled to a native DLL file, compared to the MAXScript language, which is interpreted in execution time, while also having access to several low-level functions of the application. On the other hand, MAXScript, has several UI automation functions unavailable to the C++ API. Neither the API nor the MAXScript language have access to all 3ds Max functions, meaning mixing both approaches helps leverage the software better.

With the scene files open, we begin the material conversion process itself by opening the 3ds Max Material Node Editor, selecting all materials present in the scene and creating nodes for each one of them. After that, we scan all material nodes in the scene, searching for default V–Ray materials. Then, we need to convert said default materials to V–Ray light materials. We also consider more complex layered materials.

After converting the materials, for each of the scenes objects and their respective meshes, we will apply UV unwrapping. To get better results, we will apply the custom repack modifier, in the UVW Unwrap editor window. With the unwrapping done, we can start the texture baking step, which might be an inherently lengthy process depending on the number of textures as well as their target resolutions – which we can set at the plugin UI before starting the material baking. In the UI, we can also set a padding parameter for texture dilation – a sizable enough padding will prevent the resulting baked textures from having eventual edge artifacts. Note that previously converting the V–Ray materials to V–Ray light materials is important to achieve the desired results in the texture baking step.

To view the process results in 3ds Max after baking the textures, we will convert the materials once again to standard bitmap materials, load the baked textures, update the texture UV channel and finally, update the viewport. Now the scene is ready to be exported as an FBX file, which is done automatically to a directory chosen at the beginning of the process and it’s the same folder the baked textures are saved to.

Conclusion

An early version of this plugin prototype, not including all features described here, was presented by DGG to selected participants at the Khronos Group 3D Commerce 2020 F2F Meeting in Barcelona. It was a great way to get feedback from the community, and according to the DGG team on–site, nearly everyone at the meeting had a great interest in this functionality, which is exciting!

Outlook

To wrap up, I joined my Master’s in early March, so I had to leave and couldn’t see the project to completion. A possible next step would include adding normal and bump map baking functions to the plugin, as for now, we only support diffuse V–Ray default materials. Many other interesting things could be done and it will be amazing to see where DGG takes this idea!

Thanks for reading!

Outlook from DGG

While currently the efficient use of 3D data is potentially made harder through an extra manual processing step, the plugin João worked on has the potential to solve this issue. Many retailers have high–quality 3D models available, the so-called ‘base assets’, which they use to render product imagery. Combined with RapidCompact, the transition from an offline asset to a compact real-time ready model finally becomes free of any manual work. The same models used for renderings can then be quickly processed and shown on the web or in AR/VR applications.

Find out more about the 3D content bottleneck.