search

NVlabs / Deep_Object_Pose

Deep Object Pose Estimation (DOPE) – ROS inference (CoRL 2018)
Other
1.03k stars 287 forks source link

Material is not loaded when creating custom data #263

Closed qpwodlsqp closed 2 years ago

qpwodlsqp commented 2 years ago

Hello, I have an issue with creating custom data using the script in nvisii_data_gen directory.

I made a custom model in the Blender and exported it to a .fbx to include meshes and materials in a single file. It seems that the nvisii entity should load the material in the model file when passing a file path with path_single_bj argument, but it did not load as seen in below. Can you suggest any possible reasons for this? Or shoud I explicily export meshes and textures separately?

blender_sc

nvisii_result

TontonTremblay commented 2 years ago

It should load the textures / materials. But I think there are a couple hacks in there to only use a single object. I should add a warning when there are more than one.

a lot of things happen when an object gets loaded through the import_scene (nvisii function).

https://github.com/NVlabs/Deep_Object_Pose/blob/master/scripts/nvisii_data_gen/single_video_pybullet.py#L280-L293

https://github.com/NVlabs/Deep_Object_Pose/blob/master/scripts/nvisii_data_gen/single_video_pybullet.py#L277 takes the material 0, I think you have 3 in your file.

If you send me your bx or blend file I can probably find a way to make sure files like yours work.

qpwodlsqp commented 2 years ago

Blender File FBX File Here are google drive links to my blender file and fbx. I think I should try various object hierarchies for now.

TontonTremblay commented 2 years ago

Ok I just looked into it, and I am afraid I am quite limited in terms of getting this working right now. I will start to think of a solution, but I cannot get you running quickly. With the symmetrical code we added the object have to be a single mesh. In order to color different parts differently you need to use 2d texturing. If you are using the obj coloring material like you are currently, and it is not your fault/purpose, nvisii creates different materials with different entities and the mesh gets sectioned in multiple components, and when we apply the symmetrical transformations things get weird.

Right now the quickest solution is to use a texture, https://www.youtube.com/watch?v=bHLT5Xh_tzQ, then export as an 'obj', this should fix the problem short term. Long term I need to rework the adding_mesh_object function when there is no symmetrical information and the object has multiple layers of materials. Or I need to rewrite these scripts to work with blender.

qpwodlsqp commented 2 years ago

Thank you for the tutorial link! I gonna close this issue.