Loading objects in BOP script

[NaiyerRafique]

Hi! I have a question regarding loading objects in the BOP script. I am working the BOP physics positioning script and the BOP datasets are working alright with good results. But I have been trying to load .obj and .ply objects other than the BOP dataset into the main script. I haven't been able to do that successfully. Is it possible to load such objects in the BOP physics positioning script. If yes, what changes do I need to make in the script to do so? An answer would be highly appreciated!

Best Regards

[themasterlink]

Hey,

of course this is possible, I assume you use Blenderproc 2.2.0.

How to load objects is done in this tutorial:

https://github.com/DLR-RM/BlenderProc/blob/main/docs/tutorials/loader.md

If this does not help you check or example section, there we load objects too:

https://github.com/DLR-RM/BlenderProc/tree/main/examples/basics

I hope this helps.

Best, Max

[NaiyerRafique]

I am using version 2.0.0a7 Actually I have tried that but I think I should add some detail to show the full picture.

In the BOP Challenge example in the bop_object_physics_positioning example I tried to add another object (a model of a pen) in addition to the sampled BOP objects and distractor BOP objects using the command :

extra_obj = bproc.loader.load_obj("Pen_L3.ply")

Then in the #set shading and physics properties and randomize PBR materials I changed the first line to:

for j, obj in enumerate(sampled_bop_objs + distractor_bop_objs + extra_obj):

Similarly in other places I changed it to the following:

bproc.object.sample_poses(objects_to_sample = sampled_bop_objs + distractor_bop_objs + extra_obj,

and

bop_bvh_tree = bproc.object.create_bvh_tree_multi_objects(sampled_bop_objs + distractor_bop_objs + extra_obj)

It runs successfully but somehow fails to add the "extra_object" I added other than the BOP objects in the rendered images. This happens in case of .ply models

the following error occurs in case I load an .obj file

/StructUtility.py", line 51, in get_cp
    value = self.blender_obj[key]
KeyError: 'bpy_struct[key]: key "bop_dataset_name" not found'

I downloaded these new objects from https://free3d.com/ I hope my question is clear and you have a solution for it. Best Regards

[themasterlink]

Hey,

this issue was raised before: https://github.com/DLR-RM/BlenderProc/issues/381#issuecomment-996839594

The answer in blenderproc 2.0.0 has changed a bit to:

for obj in extra_obj:
    obj.set_cp("bop_dataset_name", "my_own_dataset_name")

As the error message already said is now key set in the get_cp function. So we set one with the set_cp function, this is required as each object has to belong to a certain bop dataset, if not the writer does not work according to the bop specification.

This should resolve this issue.

Best, Max

[NaiyerRafique]

Thanks a lot! I am now able to see the extra_object in the rendered image. But I also need the BOP annotations for the extra_object. I would like to know how to change the following code to see the extra_objects added in scene_gt.json

# Write data in bop format
bproc.writer.write_bop(os.path.join(args.output_dir, 'bop_data'),
                       dataset = args.bop_dataset_name,
                       depths = bproc.postprocessing.dist2depth(data["distance"]),
                       colors = data["colors"], 
                       color_file_format = "JPEG",
                       ignore_dist_thres = 10)

Kind Regards

[themasterlink]

In BlenderProc 2.2.0:

https://github.com/DLR-RM/BlenderProc/blob/b9e751a6f27c2b233f6f4716209c087230cfc535/blenderproc/python/writer/BopWriterUtility.py#L19-L26

This should give you the option to save the type of objects you want to save.

[NaiyerRafique]

I have a couple of questions.

1. When I import a BOP dataset as a distractor object, there are no problems but if I import them through the following command:

extra_obj = bproc.loader.load_obj("BOP_Datasets/hb/models_eval/obj_000032.ply") I get the following message:

Warning: Detected that obj_000007 is completely inside obj_000032. This might be wrong, if obj_000032 is not water tight or has incorrect normals. If that is the case, consider setting skip_inside_check to True.

This happens whether I use the BOP dataset or any random model. Although the rendering completes successfully, the images are a complete mess.

2. After testing a lot of objects and getting the same result, I was able to load one random model which wasn't a BOP object. Then I tried to obtain poses for this object through the target_objects argument of the bproc.writer.write_bop function. I applied the following commands:

obj_list = extra_obj + sampled_bop_objs

then I applied target_objects = obj_list and I got the following error:

assert "category_id" in obj, "{} object has no custom property 'category_id'".format(obj.get_name()) AttributeError: 'Object' object has no attribute 'get_name'

Error: script failed, file: 'examples/datasets/bop_object_physics_positioning/mainpractice.py', exiting. But if I simply put target_objects = sampled_bop_objs or target_objects = distractor_bop_objs, I get the poses for both the sampled and distractor objects. The error only shows when I use a random object.

Are there any solutions for these problems?

Best Regards

[MartinSmeyer]

Hi @NaiyerRafique

1.) Loaded objects are initially placed at the origin. Therefore it can happen that an object is completely inside of another object. This is what the warning in bproc.loader.load_obj() is telling you. The bproc.loader.load_bop_objs() function does not have this warning. As long as you move the objects later on, you can ignore that warning.

What is messing up your rendering is the fact that BOP model vertices are defined in mm. If you set bproc.loader.load_bop_objs(.., mm2m=True) the BOP models are automatically scaled to meters. However if you just use bproc.loader.load_obj() it will still be in mm, therefore you are most likely rendering inside a huge 3D model. You can scale them to meters by extra_obj.set_scale([0.001, 0.001, 0.001]). Please have a look at this basic example to load a BOP object with bproc.loader.load_obj(): https://github.com/DLR-RM/BlenderProc/blob/main/examples/basics/camera_object_pose/main.py

2.) The bproc.loader.load_bop_objs() is taking care of setting the right category_id, i.e. the BOP object id. If you load the BOP object with bproc.loader.load_obj() you need to do it yourself extra_obj.set_cp("category_id", 42), again have a look at the example I linked above