Unstable physics after spawning on surface

[JanGlinko]

Describe the bug After spawning items using function blenderproc.object.sample_poses_on_surface physics simulation seems unstable.

General Information

1. Which BlenderProc version are you using? 2.1.0
2. On which operating system are you? Ubuntu 20.04
3. Have you checked the issue tracker to see if a similar issue has been opened? Yes
4. Have you changed BlenderProc in any way besides the config file? If yes, are you sure that this change does not affect the problem you are having? No To Reproduce Steps to reproduce the behavior:
5. Provide the full command you used to run BlenderProc: blenderproc run generate_dataset.py dataset
6. Provide the full python file, you used:

   
   import json
   import math
   import numpy as np
   import argparse

parser = argparse.ArgumentParser() parser.add_argument('output', nargs='?') args = parser.parse_args()

def get_all_items_list(path_json: str) -> dict: with open(path_json, 'r') as f: return json.load(f)

def choose_items_to_load(items: dict, n: int) -> (list, list): ids = list(set(items.values())) ids_list = list(np.random.choice(ids, n)) ids_list = list(map(int, ids_list))

items_list = []
for chosen_id in ids_list:
    matching_items = [k for k, v in items.items() if v == chosen_id]
    items_list.append(np.random.choice(matching_items, 1)[0])

return items_list, ids_list

def sample_pose_wrapper(obj_parent: blenderproc.types.MeshObject, d1_max, d2_max): def sample_pose_inside(obj_sampled_inside): obj_sampled_inside.set_location(blenderproc.sampler.upper_region( objects_to_sample_on=obj_parent, min_height=0.2, max_height=0.4, use_ray_trace_check=True, upper_dir=[0.0, 0.0, 1.0], use_upper_dir=True )) obj_sampled_inside.set_rotation_euler(np.random.uniform([0, 0, 0], [d1_max, d2_max, np.pi * 2]))

return sample_pose_inside

def main(): n = 1 all_items = get_all_items_list("/home/avena/software/items3/loading_dictionaries/items_dictionary.json") consumables_items = get_all_items_list("/home/avena/software/items3/loading_dictionaries/consumables_dictionary.json") containers_items = get_all_items_list("/home/avena/software/items3/loading_dictionaries/containers_dictionary.json") plane_containers = {} capable_containers = {} for k, v in containers_items.items(): if "bowl" in k or "plate" in k: capable_containers[k] = v else: plane_containers[k] = v

for _ in range(n):
    scenario_number = np.random.randint(0, 1)

    blenderproc.init()
    blenderproc.utility.reset_keyframes()

    table = blenderproc.loader.load_blend("/home/avena/software/items3/scenes/Box_stol.blend")[0]
    table.enable_rigidbody(False, collision_shape='MESH')

    if scenario_number == 0:
        number_of_items = np.random.randint(5, 20)
        items_to_load, ids_of_items_to_load = choose_items_to_load(all_items, number_of_items)

        sampler = sample_pose_wrapper(table, 0, 0)
        loaded_items = [item for item_to_load in items_to_load for item in
                        blenderproc.loader.load_obj(item_to_load)]
        for item, item_id, name in zip(loaded_items, ids_of_items_to_load, items_to_load):
            item.set_cp("category_id", item_id)
            item_name = name.split("/")[-3].split("-")[-1]
            item.set_name(item_name)
            item.enable_rigidbody(True, collision_shape='MESH')

        blenderproc.object.sample_poses_on_surface(loaded_items, table, sampler, min_distance=0.1, max_distance=3)

        blenderproc.object.simulate_physics_and_fix_final_poses(min_simulation_time=0.5, max_simulation_time=4,
                                                                check_object_interval=1, substeps_per_frame=10)
    if scenario_number == 1:
        pass
    if scenario_number == 2:
        pass

    light_p = blenderproc.types.Light()
    light_p.set_type("POINT")
    r = np.random.randint(2, 10)
    alpha = np.random.randint(0, 360)
    y = r * math.sin(math.pi / 180 * alpha)
    x = r * math.cos(math.pi / 180 * alpha)
    z = np.random.randint(2, 5)
    light_p.set_location([x, y, z])
    light_p.set_energy(1000)

    blenderproc.camera.set_intrinsics_from_blender_params(lens=0.017453, image_width=2400, image_height=1350,
                                                          lens_unit='FOV')

    position = [0, 0, 138]
    rotation = [0, 0, 0]
    matrix_world = blenderproc.math.build_transformation_mat(position, rotation)
    blenderproc.camera.add_camera_pose(matrix_world)

    blenderproc.renderer.set_samples(30)
    data = blenderproc.renderer.render()
    seg_data = blenderproc.renderer.render_segmap(map_by=["class", "instance", "name"])
    blenderproc.writer.write_coco_annotations(args.output,
                                              instance_segmaps=seg_data["instance_segmaps"],
                                              instance_attribute_maps=seg_data["instance_attribute_maps"],
                                              colors=data["colors"],
                                              color_file_format="JPEG",
                                              append_to_existing_output=True)

if name == 'main': main()

3. Provide a link to all 3D models you used, if they are from one of the publicly available supported datasets, provide the name or path so that it is possible to reproduce the error.
Not public.

**Expected behavior**
In my opinion, physics simulation should run smoothly. Probably items slightly overlap with surface, which causes unstable simulation.

**Screenshots**
First frame:
![Screenshot from 2021-12-15 09-58-48](https://user-images.githubusercontent.com/94356189/146155787-57eeb5ed-a29c-481a-97eb-b641e822e18d.png)
After few frames:
![Screenshot from 2021-12-15 09-59-08](https://user-images.githubusercontent.com/94356189/146155831-7015fd29-9c16-464d-a05c-e87c7eca13cf.png)

![Screenshot from 2021-12-15 09-59-27](https://user-images.githubusercontent.com/94356189/146155836-92bba451-abbc-4877-9d71-19faaeed73c2.png)

[cornerfarmer]

Hey @JanGlinko,

it seems you are using MESH collision_shapes. Although these collision shapes are very exact, they are also very unstable. I would therefore recommend you to use "CONVEX_HULL" collision shapes instead. If you have objects that are very non-convex, you can also use convex decomposition to get more exact but still stable collision shapes.

See this tutorial for more details: https://github.com/DLR-RM/BlenderProc/blob/main/docs/tutorials/physics.md#the-collision_shape-parameter

Let me know if that fixes your problem

[JanGlinko]

Thank You for the response. Its little more stable. But still, in the first frame all items "jumps" a little. Should i change some physics parameters like mass, friction and dumping?

[cornerfarmer]

So the first frame is before physics positioning and the second one is afterwards or what exactly do the screen shots show? And you are now using CONVEX_HULL collision shapes, right?

Can you try loading only one object and see if the positioning works then?

[JanGlinko]

Yes, the first frame is just after spawning on the surface and second one is just after running physics simulation. Yes, im using CONVEX_HULL. I think there is a problem with my models, because some of them are stable and some not

[cornerfarmer]

If you want, you can upload one model which does not work and I can take a look. Apart from that, you could run your script in debug mode and then replace simulate_physics_and_fix_final_poses( with simulate_physics(. In this way, you should be able to see the full simulation in the blender GUI and not only the final fixed poses.

[JanGlinko]

Sure, https://drive.google.com/drive/folders/1FRwKaQh2tGykUb5fXRh1vAJUdQArs2MH?usp=sharing Thank You.

[cornerfarmer]

Hey @JanGlinko,

I cannot really reproduce this behaviour. In my test, the apple is placed nicely on the table:

I used your code, just replaced "MESH" with "CONVEX_HULL" collision shapes and used a simple cube as table (as I do not have Box_stol.blend):

        table = blenderproc.object.create_primitive("CUBE")
        table.set_scale([1, 1, 0.1])

Maybe there is something wrong with your table?

[JanGlinko]

I was analyzing simulation of physics step by step and i have found that some objects' origins are not aligned with objects, and it makes simulation unstable. Creating table with Your code helped with the jumping. I also set collision_margin=0 Thanks!

[cornerfarmer]

Yeah we know about the instabilities of misplaced object origins. Thats we shift origins to the center of mass before the simulation: https://github.com/DLR-RM/BlenderProc/blob/main/blenderproc/python/object/PhysicsSimulation.py#L95 This is then undone after the simulation.