DLR-RM / BlenderProc

A procedural Blender pipeline for photorealistic training image generation
GNU General Public License v3.0
2.79k stars 449 forks source link

Option to insert frames for Material properties when possible #965

Open njsymanz opened 1 year ago

njsymanz commented 1 year ago

Describe your feature request

I am generating training data for DL, and rather than storing thousands of images, I would like to save a .blend file where the data for each image to be rendered is stored in frames. Since I am randomizing the materials of the objects, I need to be able to insert frames for each one.

As a temporary solution, I modified Material.set_principled_shader_value() to accept a "frame" argument similar to some of the methods in the Entity class. This takes care of the cases where I only need a uniform material, but ideally I would be able to apply random texture images (such as from cctextures) on individual frames. Similarly, being able to set a world background in a frame would be useful as well.

Describe a possible solution

For texture images, it doesn't seem like replacing the image can be keyframed. With my very limited knowledge of Blender and what I could scrape together from my Google searches, it seems the common workaround is to use nested Mix shaders and adjust the mix factors between 0 and 1 to select the visible texture.

cornerfarmer commented 1 year ago

As you say, unfortunately its not possible to set keyframes for textures. Why do you want to store your data into blend files? In the end, when training your neural networks, you need to render the images anyway right?

njsymanz commented 1 year ago

I'm hoping by separating the process this way I can optimize the individual parts a little better. Blend files also don't take as much space with more images, and it allows me to make adjustments to some parts of the scene without redoing all the physics simulation later on.