search

DLR-RM / BlenderProc

A procedural Blender pipeline for photorealistic training image generation
GNU General Public License v3.0
2.78k stars 449 forks source link

Get labels for texture paint or procedural textures #181

Closed CreateInsaneStuff closed 3 years ago

CreateInsaneStuff commented 3 years ago

Hi together, I was trying to generate data for my CV application. In my model, I want to simulate surface imperfetions (such as cracks) as displayed in the sample image below. Currently, I model them with nodes as procedural textures. These cracks should also have a label (as objects with coco annotations). However, as they are no objects in blender, they can't be exported with the current version. I was thinking of possible ways (besides modeling the cracks as real objects) but couldn't find one. As procedural textures are calculated by blender, there has to be at least a way to identiy them in a picture, what do you think? Thanks for your input an best regards!

sample

themasterlink commented 3 years ago

Hey,

what you could do is change the coloring step in the semantic segmentation:

https://github.com/DLR-RM/BlenderProc/blob/fe5ea258acb9e3ec11e0d4c6f380321e01c4e751/src/utility/SegMapRendererUtility.py#L57

Here, we change the color of each object to a unique color for the rendering, after that we then assign each color to a class label or an instance label. You could add something here to add two colors for these two materials, so that you could latter then separate them.

What do you think?

Best, Max

CreateInsaneStuff commented 3 years ago

Ok thanks, that sounds possible! It might take me some time until I get it running. If you want, you can close the issue meanwhile and I comment later on. Best regards and thanks!

themasterlink commented 3 years ago

Sounds good, if possible you can in the end open a PR, so that other can profit from it, too :)

CreateInsaneStuff commented 3 years ago

Hey, I tried to realize the segmentation by material, but: To model such details, I can't just assign different materials to faces or vertices. What I do (and that is as far as I know the only/best way in blender) is I mix different materials (shaders) with a grayscale mask as displayed in the image. So that it affects pixels and not faces. In the end, I only have one Material output and thus only one material for the entire object to colorize. Also if I would do it procedurally with e.g. a noise texture instead of a texture painted image, I still have only one material output in the end.

Or am I just thinking worng?

ShaderNodeSetup

@themasterlink : can we reopen the issue?