Doly enables creators and online sellers to bring their products to life by quickly generating professional-looking 3D visuals directly from the app, without any technical skills.
Capture your product
Choose a 3D video template
Receive and export your Doly
Capture
Introduction
We designed Doly with different capture modes to introduce users with no prior scan experience to in the best way possible.
In the case of Doly, creating 3D templates were a bit more involved than typical product scenes:
The product is not present in the scene and is added through compositing.
Scenes have to render fast because we don’t want users to wait a long time to receive their Dolies.
The product quality heavily depends on the quality of the capture by the user using the Doly app.
Because the product is not in the scene, it cannot move, rotate, scale, benefit from the scene’s lighting nor occlusion from other elements. Knowing these limitations and how they would impact the quality and variety of 3D templates, we worked hard to lift many of these through better compositing and enhancements to the pipe over time.
After implementing these enhancements, the product is still not in the scene but it can move, rotate and even be occluded by other elements in the scene. While the user’s product is added through compositing, we also insert an invisible proxymesh, an approximate mesh of the product we create after the capture, to generate the product’s shadow and reflection.
Starting March 2024, Mattéo Mérand, a scenography student, joined the team as our intern and helped me creating templates. Mattéo’s skills improved significantly to the point he became completely autonomous and helped the team lifting a lot of limits I wrote about earlier.
Template Manager tool
We needed a way to manage templates in the gallery as we created more and we also wanted to easily create thematic categories. We created an internal tool called the Template Manager allowing us to manage both DEV and PROD environments.
I designed it so we could:
Easily move templates from one category to the other.
Edit a template’s metadata and change its assets shown in the app’s gallery.
Push a template from the DEV environment to PROD.
Hide or a show entire categories or individual templates in the app’s gallery.
Templates
Introduction
Blender was chosen as the 3D software to create templates to place the user’s products in. Blender is free, open source, has a vibrant community, a strong ecosystem of add-ons and can run headless on many configurations, including the pipe.
In the case of Doly, creating 3D templates were a bit more involved than typical product scenes:
The product is not present in the scene and is added through compositing.
Scenes have to render fast because we don’t want users to wait a long time to receive their Dolies.
The product quality heavily depends on the quality of the capture by the user using the Doly app.
Because the product is not in the scene, it cannot move, rotate, scale, benefit from the scene’s lighting nor occlusion from other elements. Knowing these limitations and how they would impact the quality and variety of 3D templates, we worked hard to lift many of these through better compositing and enhancements to the pipe over time.
After implementing these enhancements, the product is still not in the scene but it can move, rotate and even be occluded by other elements in the scene. While the user’s product is added through compositing, we also insert an invisible proxymesh, an approximate mesh of the product we create after the capture, to generate the product’s shadow and reflection.
Starting March 2024, Mattéo Mérand, a scenography student, joined the team as our intern and helped me creating templates. Mattéo’s skills improved significantly to the point he became completely autonomous and helped the team lifting a lot of limits I wrote about earlier.
Creating a template
We divided templates into two depending on the capture mode:
180 Template for Quick 180 and 180 captures.
360 Template for 360 captures.
A Doly blender template consists of the following elements:
Normalization_Box: A cube empty acting as a placeholder for the user’s product.
Cameras.
Lights.
Any other elements you would create or expect in a 3D scene.
While the Normalization_Box is enough to build a scene, it’s a bit tricky to define cameras and what they’re looking at without the product and since the product’s quality is heavily dependent on the capture’s quality, it means we need to be careful with camera angles to prevent showing holes or issues if the capture is not great.
To help, we import a proxymesh which gives us a good indication of what the camera should look at and better define angles.
Testing a template
For some time, the only way to test a template was to send it to the pipe and cross my fingers everything would render correctly without breaking the pipe. Then, the engineering team would provide me with dockr image so I could test and iterate on templates directly on my PC using the Linux subsystem.
Later, we created an internal tool allowing us to upload templates, render one or multiple test Dolies and generate the necessary assets to push the templates to the gallery.
Template Manager tool
We needed a way to manage templates in the gallery as we created more and we also wanted to easily create thematic categories. We created an internal tool called the Template Manager allowing us to manage both DEV and PROD environments.
I designed it so we could:
Easily move templates from one category to the other.
Edit a template’s metadata and change its assets shown in the app’s gallery.
Push a template from the DEV environment to PROD.
Hide or a show entire categories or individual templates in the app’s gallery.
TemplateGEN tool
We wanted to explore GenAI to quickly create more variety of templates and we wanted to try to automate it as much as possible. As a proof of concept, I manually went through the following:
Create one blender file containing the cameras, some lights and a slightly curved image plane for the environment in the background.
Generate an image using stable diffusion and import it in the curved image plane.
Create a support in its own blender file.
Create one more blender file containing the cameras, the image plane, the support and the normalization_box with the necessary compositing so the user’s product shows up.
After a few iterations and making sure that everything comes together nicely, we decided to make it a tool with built-in stable diffusion image generation and a way to immediately test and publish the template to the DEV environment.
I designed it so we could:
Select a base template blender file containing cameras trajectories, some lights and the normalization_box.
Generate as many images as we want using Fooocus and automatically import it as the curved image plane of the base template blender file.
Select one support I created to place the product on and import it to the template file.
Generate a Doly preview.
Publish the template to the DEV environment and the template manager.
While the generated template looks fine, it’s sometimes necessary to manually adjust its lights so it looks better.