Closed b-yogesh closed 3 years ago
Hey @b-yogesh,
I think there are two ways to get BlenderProc to work with Phobos:
For controlling your robot I would advice you to write your own module (its basically just a python class). In this way its easier to implement more advanced behaviour. Also, it seems to me that the loaded robots are controlled via bones, whose manipulation is not yet supported by the entity manipulator.
Hey @cornerfarmer ,
Thank you very much for your prompt reply. I was trying your first suggestion yesterday (Phobos work with blender 2.9). So, I was able to link Phobos + Blender 2.90 + BlenderProc with a few changes in the script of Phobos. I am able to load the URDF file and manipulate using Rotate function of Blender GUI. It all seemed fine until I ran the basic example of BlenderProc.
This is the image from the Blender RenderImage:
And this is the one that BlenderProc generates:
Based on ablation study of the URDF model, when I removed the links of the robot, it gave me the exact image as the BlenderProc.
This is the URDF robot structure:
Check out these logs:
Here, it does not load any Links into BlenderProc which causes all the meshes to collapse to the origin, which I suspect is the issue. Can you please help me out here as how do I make this work?
Hey @b-yogesh,
thanks for the detailed description.
The log you added only contains the rgb renderer, I am not sure if links are explicitly listed here. However, you are probably right, that something in the loading of your robot goes wrong. Can you please explain how you load your URDF model using BlenderProc. Did you write your own module? Which functions from the Phobos extension do you call?
Hello,
Phobos has a built-in functionality to import URDF models into it. I used that.
Have I made any mistake? Because, while importing from phobos, the links are imported and I am able to manipulate them using rotation. But, the links do not appear in the BlenderProc output 😞
Here are the complete logs:
_Blender 2.90.0 (hash 0330d1af29c0 built 2020-08-31 10:00:13) Read prefs: C:\Users\Yogesh Kumar\AppData\Roaming\Blender Foundation\Blender\2.90\config\userpref.blend found bundled python: J:\SaarlandUniversity\MasterThesis\BlenderProc\blender-2.90.0-windows64\2.90\python Importing phobos IMPORT: phobos.defs Parsing definitions from: C:\Users\Yogesh Kumar\AppData\Roaming\phobos/definitions defaultControllers.yml C:\Users\Yogesh Kumar\AppData\Roaming\Blender Foundation\Blender\2.90\scripts\addons\phobos\defs.py:201: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. tmpyaml = yaml.load(evaluateString(tmpstring)) defaultJoints.yml defaultMaterials.yml defaultModel.yml defaultMotors.yml defaultSensors.yml defaultSubmechanisms.yml defaultSubmodels.yml Creating new definition type: joints IMPORT: phobos.display IMPORT: phobos.io Registered mesh plugin: ['obj', 'stl', 'dae'] ERROR in scenes/init: "mars.py" has no valid scene plugin interface. Registered scene plugin: ['sdf world'] Registered scene plugin: ['smurfs'] Registered entity plugin: ['generic'] ERROR in entities/init: "heightmap.py" has no valid entity plugin interface. Registered entity plugin: ['light'] Registered entity plugin: ['primitive'] Registered entity plugin: ['sdf'] Registered entity plugin: ['smurf'] Registered entity plugin: ['srdf'] Registered entity plugin: ['submechanisms'] Registered entity plugin: ['thumbnails'] Registered entity plugin: ['urdf'] Registered entity plugin: ['yaml'] Registered mesh plugin: ['obj', 'stl', 'dae'] ERROR in scenes/init__: "mars.py" has no valid scene plugin interface. Registered scene plugin: ['sdf world'] Registered scene plugin: ['smurfs'] RELOAD: phobos.model IMPORT: phobos.operators IMPORT: phobos.phobosgui RELOAD: phobos.phoboslog RELOAD: phobos.phobossystem RELOAD: phobos.utils Registering operators.selection... Registering operators.io... Registering operators.editing... Registering operators.naming... Registering operators.misc...
Registering phobosgui... ... successful. Info: Deleted 3 object(s) Info: Deleted 3 object(s)
Device GeForce 940MX (Display) of type OPTIX found and used.
Resolution: 1280, 720 Fra:0 Mem:58.80M (Peak 62.14M) | Time:00:00.21 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | visual_0_kuka_arm_7_link Fra:0 Mem:58.81M (Peak 62.14M) | Time:00:00.21 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | visual_0_kuka_arm_6_link Fra:0 Mem:58.88M (Peak 62.14M) | Time:00:00.21 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | visual_0_kuka_arm_5_link Fra:0 Mem:58.96M (Peak 62.14M) | Time:00:00.21 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | visual_0_kuka_arm_4_link Fra:0 Mem:59.07M (Peak 62.14M) | Time:00:00.21 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | visual_0_kuka_arm_3_link Fra:0 Mem:59.17M (Peak 62.14M) | Time:00:00.21 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | visual_0_kuka_arm_2_link Fra:0 Mem:59.27M (Peak 62.14M) | Time:00:00.21 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | visual_0_kuka_arm_1_link Fra:0 Mem:59.38M (Peak 62.14M) | Time:00:00.21 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | visual_0_calib_kuka_arm_base_link Fra:0 Mem:59.42M (Peak 62.14M) | Time:00:00.22 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | resource::joint_revolute Fra:0 Mem:59.49M (Peak 62.14M) | Time:00:00.22 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | resource::joint_revolute.001 Fra:0 Mem:59.57M (Peak 62.14M) | Time:00:00.22 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | resource::joint_revolute.002 Fra:0 Mem:59.65M (Peak 62.14M) | Time:00:00.22 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | resource::joint_revolute.003 Fra:0 Mem:59.72M (Peak 62.14M) | Time:00:00.22 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | resource::joint_revolute.004 Fra:0 Mem:59.80M (Peak 62.14M) | Time:00:00.23 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | resource::joint_revolute.005 Fra:0 Mem:59.87M (Peak 62.14M) | Time:00:00.23 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | resource::joint_revolute.006 Fra:0 Mem:59.95M (Peak 62.14M) | Time:00:00.23 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | resource::joint_revolute.007 Fra:0 Mem:60.03M (Peak 62.14M) | Time:00:00.23 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | inertial_kuka_arm_7_link Fra:0 Mem:60.03M (Peak 62.14M) | Time:00:00.23 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | inertial_kuka_arm_6_link Fra:0 Mem:60.03M (Peak 62.14M) | Time:00:00.23 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | inertial_kuka_arm_5_link Fra:0 Mem:60.03M (Peak 62.14M) | Time:00:00.24 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | inertial_kuka_arm_4_link Fra:0 Mem:60.03M (Peak 62.14M) | Time:00:00.24 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | inertial_kuka_arm_3_link Fra:0 Mem:60.03M (Peak 62.14M) | Time:00:00.24 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | inertial_kuka_arm_2_link Fra:0 Mem:60.03M (Peak 62.14M) | Time:00:00.24 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | inertial_kuka_arm_1_link Fra:0 Mem:60.03M (Peak 62.14M) | Time:00:00.24 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | inertial_calib_kuka_arm_base_link Fra:0 Mem:60.04M (Peak 62.14M) | Time:00:00.24 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | Icosphere Fra:0 Mem:60.04M (Peak 62.14M) | Time:00:00.24 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Synchronizing object | Cube Fra:0 Mem:60.04M (Peak 62.14M) | Time:00:00.24 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Initializing Fra:0 Mem:56.82M (Peak 62.14M) | Time:00:00.25 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Waiting for render to start Fra:0 Mem:56.82M (Peak 62.14M) | Time:00:00.25 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Loading render kernels (may take a few minutes the first time) Fra:0 Mem:56.82M (Peak 63.74M) | Time:00:00.89 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Updating Scene Fra:0 Mem:56.82M (Peak 63.74M) | Time:00:00.90 | Mem:0.00M, Peak:0.00M | Scene, View Layer | Updating Shaders Fra:0 Mem:57.33M (Peak 63.74M) | Time:00:01.05 | Mem:0.01M, Peak:0.01M | Scene, View Layer | Updating Background Fra:0 Mem:57.33M (Peak 63.74M) | Time:00:01.05 | Mem:0.01M, Peak:0.01M | Scene, View Layer | Updating Camera Fra:0 Mem:57.33M (Peak 63.74M) | Time:00:01.05 | Mem:0.01M, Peak:0.01M | Scene, View Layer | Updating Meshes Flags Fra:0 Mem:57.33M (Peak 63.74M) | Time:00:01.05 | Mem:0.01M, Peak:0.01M | Scene, View Layer | Updating Objects Fra:0 Mem:57.33M (Peak 63.74M) | Time:00:01.06 | Mem:0.01M, Peak:0.01M | Scene, View Layer | Updating Objects | Copying Transformations to device Fra:0 Mem:57.33M (Peak 63.74M) | Time:00:01.06 | Mem:0.01M, Peak:0.01M | Scene, View Layer | Updating Objects | Applying Static Transformations Fra:0 Mem:57.33M (Peak 63.74M) | Time:00:01.06 | Mem:0.01M, Peak:0.01M | Scene, View Layer | Updating Particle Systems Fra:0 Mem:57.33M (Peak 63.74M) | Time:00:01.06 | Mem:0.01M, Peak:0.01M | Scene, View Layer | Updating Particle Systems | Copying Particles to device Fra:0 Mem:57.33M (Peak 63.74M) | Time:00:01.06 | Mem:0.01M, Peak:0.01M | Scene, View Layer | Updating Meshes Fra:0 Mem:57.60M (Peak 63.74M) | Time:00:01.06 | Mem:0.01M, Peak:0.01M | Scene, View Layer | Updating Mesh | Computing attributes Fra:0 Mem:57.62M (Peak 63.74M) | Time:00:01.06 | Mem:0.01M, Peak:0.01M | Scene, View Layer | Updating Mesh | Copying Attributes to device Fra:0 Mem:57.61M (Peak 63.74M) | Time:00:01.07 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Cube.009 26/26 | Building BVH Fra:0 Mem:57.62M (Peak 63.74M) | Time:00:01.07 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Icosphere 25/26 | Building BVH Fra:0 Mem:57.62M (Peak 63.74M) | Time:00:01.07 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Cube.042 24/26 | Building BVH Fra:0 Mem:57.62M (Peak 63.74M) | Time:00:01.07 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Cube.043 23/26 | Building BVH Fra:0 Mem:57.62M (Peak 63.74M) | Time:00:01.07 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Cube.044 22/26 | Building BVH Fra:0 Mem:57.62M (Peak 63.74M) | Time:00:01.07 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Cube.045 21/26 | Building BVH Fra:0 Mem:57.62M (Peak 63.74M) | Time:00:01.07 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Cube.046 20/26 | Building BVH Fra:0 Mem:57.62M (Peak 63.74M) | Time:00:01.07 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Cube.047 19/26 | Building BVH Fra:0 Mem:57.63M (Peak 63.74M) | Time:00:01.07 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Cube.048 18/26 | Building BVH Fra:0 Mem:57.63M (Peak 63.74M) | Time:00:01.08 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Cube.049 17/26 | Building BVH Fra:0 Mem:57.63M (Peak 63.74M) | Time:00:01.08 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH clockwise_turnaxis.008 16/26 | Building BVH Fra:0 Mem:57.70M (Peak 63.74M) | Time:00:01.08 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH clockwise_turnaxis.007 15/26 | Building BVH Fra:0 Mem:57.77M (Peak 63.74M) | Time:00:01.08 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH clockwise_turnaxis.006 14/26 | Building BVH Fra:0 Mem:57.84M (Peak 63.74M) | Time:00:01.08 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH clockwise_turnaxis.005 13/26 | Building BVH Fra:0 Mem:57.91M (Peak 63.74M) | Time:00:01.08 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH clockwise_turnaxis.004 12/26 | Building BVH Fra:0 Mem:57.98M (Peak 63.74M) | Time:00:01.08 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH clockwise_turnaxis.003 11/26 | Building BVH Fra:0 Mem:58.05M (Peak 63.74M) | Time:00:01.09 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH clockwise_turnaxis.002 10/26 | Building BVH Fra:0 Mem:58.12M (Peak 63.74M) | Time:00:01.09 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH clockwise_turnaxis.001 9/26 | Building BVH Fra:0 Mem:58.19M (Peak 63.74M) | Time:00:01.09 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Arm Base.001 8/26 | Building BVH Fra:0 Mem:58.23M (Peak 63.74M) | Time:00:01.09 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Arm Segment A.002 7/26 | Building BVH Fra:0 Mem:58.33M (Peak 63.74M) | Time:00:01.09 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Arm Segment B.002 6/26 | Building BVH Fra:0 Mem:58.43M (Peak 63.74M) | Time:00:01.09 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Arm Segment A.003 5/26 | Building BVH Fra:0 Mem:58.53M (Peak 63.74M) | Time:00:01.09 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Arm Segment B.003 4/26 | Building BVH Fra:0 Mem:58.63M (Peak 63.74M) | Time:00:01.09 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Arm Segment Last.001 3/26 | Building BVH Fra:0 Mem:58.70M (Peak 63.74M) | Time:00:01.10 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Arm Wrist.001 2/26 | Building BVH Fra:0 Mem:58.77M (Peak 63.74M) | Time:00:01.10 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Geometry BVH Arm Flanche.001 1/26 | Building BVH Fra:0 Mem:58.77M (Peak 63.74M) | Time:00:01.10 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Scene BVH | Building Fra:0 Mem:59.93M (Peak 63.74M) | Time:00:01.10 | Mem:0.02M, Peak:0.02M | Scene, View Layer | Updating Scene BVH | Copying BVH to device Fra:0 Mem:59.93M (Peak 63.74M) | Time:00:01.11 | Mem:1.18M, Peak:1.18M | Scene, View Layer | Updating Scene BVH | Building OptiX acceleration structure Fra:0 Mem:60.00M (Peak 63.74M) | Time:00:01.16 | Mem:1.18M, Peak:1.27M | Scene, View Layer | Updating Mesh | Computing normals Fra:0 Mem:60.65M (Peak 63.74M) | Time:00:01.16 | Mem:1.18M, Peak:1.27M | Scene, View Layer | Updating Mesh | Copying Mesh to device Fra:0 Mem:60.58M (Peak 63.74M) | Time:00:01.16 | Mem:1.83M, Peak:1.83M | Scene, View Layer | Updating Objects Flags Fra:0 Mem:60.58M (Peak 63.74M) | Time:00:01.16 | Mem:1.83M, Peak:1.83M | Scene, View Layer | Updating Images Fra:0 Mem:60.58M (Peak 63.74M) | Time:00:01.16 | Mem:1.83M, Peak:1.83M | Scene, View Layer | Updating Camera Volume Fra:0 Mem:60.58M (Peak 63.74M) | Time:00:01.16 | Mem:1.83M, Peak:1.83M | Scene, View Layer | Updating Lookup Tables Fra:0 Mem:60.58M (Peak 63.74M) | Time:00:01.17 | Mem:2.08M, Peak:2.08M | Scene, View Layer | Updating Lights Fra:0 Mem:60.59M (Peak 63.74M) | Time:00:01.17 | Mem:2.08M, Peak:2.08M | Scene, View Layer | Updating Lights | Computing distribution Fra:0 Mem:60.59M (Peak 63.74M) | Time:00:01.17 | Mem:2.08M, Peak:2.08M | Scene, View Layer | Updating Integrator Fra:0 Mem:61.85M (Peak 63.74M) | Time:00:01.18 | Mem:3.35M, Peak:3.35M | Scene, View Layer | Updating Film Fra:0 Mem:61.85M (Peak 63.74M) | Time:00:01.18 | Mem:3.10M, Peak:3.35M | Scene, View Layer | Updating Lookup Tables Fra:0 Mem:61.85M (Peak 63.74M) | Time:00:01.18 | Mem:3.35M, Peak:3.35M | Scene, View Layer | Updating Baking Fra:0 Mem:61.85M (Peak 63.74M) | Time:00:01.18 | Mem:3.35M, Peak:3.35M | Scene, View Layer | Updating Device | Writing constant memory Fra:0 Mem:61.85M (Peak 63.74M) | Time:00:01.18 | Mem:3.35M, Peak:3.35M | Scene, View Layer | Updating Device | Writing constant memory | Compiling render kernels Fra:0 Mem:61.85M (Peak 63.74M) | Time:00:01.19 | Mem:3.35M, Peak:3.35M | Scene, View Layer | Updating Device | Writing constant memory Fra:0 Mem:61.86M (Peak 63.74M) | Time:00:01.19 | Mem:3.35M, Peak:3.35M | Scene, View Layer | Rendered 0/15 Tiles, Sample 0/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:02.21 | Remaining:00:05.71 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 0/15 Tiles, Sample 328/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:02.29 | Remaining:00:06.39 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 1/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:03.22 | Remaining:00:09.77 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 1/15 Tiles, Sample 304/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:03.37 | Remaining:00:10.01 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 2/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:04.23 | Remaining:00:10.46 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 2/15 Tiles, Sample 280/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:04.45 | Remaining:00:10.49 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 3/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:05.24 | Remaining:00:10.29 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 3/15 Tiles, Sample 256/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:05.54 | Remaining:00:10.19 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 4/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:06.25 | Remaining:00:09.79 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 4/15 Tiles, Sample 232/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:06.62 | Remaining:00:09.57 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 5/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:07.26 | Remaining:00:09.12 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 5/15 Tiles, Sample 208/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:07.70 | Remaining:00:08.81 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 6/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:08.28 | Remaining:00:08.80 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 6/15 Tiles, Sample 120/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:09.28 | Remaining:00:08.66 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 6/15 Tiles, Sample 328/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:09.40 | Remaining:00:08.65 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 7/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:10.31 | Remaining:00:09.11 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 7/15 Tiles, Sample 80/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:11.32 | Remaining:00:09.53 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 7/15 Tiles, Sample 168/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:12.32 | Remaining:00:09.86 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 7/15 Tiles, Sample 256/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:13.33 | Remaining:00:10.11 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 7/15 Tiles, Sample 344/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:13.41 | Remaining:00:10.13 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 8/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:14.34 | Remaining:00:08.65 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 8/15 Tiles, Sample 304/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:14.49 | Remaining:00:08.44 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 9/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:15.35 | Remaining:00:07.17 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 9/15 Tiles, Sample 280/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:15.57 | Remaining:00:06.87 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 10/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:16.36 | Remaining:00:05.78 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 10/15 Tiles, Sample 256/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:16.66 | Remaining:00:05.39 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 11/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:17.37 | Remaining:00:04.44 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 11/15 Tiles, Sample 232/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:17.74 | Remaining:00:03.97 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 12/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:18.38 | Remaining:00:03.16 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 12/15 Tiles, Sample 208/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:18.83 | Remaining:00:02.61 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 13/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:19.39 | Remaining:00:01.91 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 13/15 Tiles, Sample 184/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:19.91 | Remaining:00:01.29 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 14/15 Tiles, Sample 350/350 Fra:0 Mem:65.61M (Peak 69.36M) | Time:00:20.40 | Remaining:00:00.69 | Mem:7.10M, Peak:7.10M | Scene, View Layer | Rendered 14/15 Tiles, Sample 160/350 Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:20.99 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Rendered 15/15 Tiles Fra:0 Mem:61.86M (Peak 69.36M) | Time:00:20.99 | Mem:3.35M, Peak:7.10M | Scene, View Layer | Finished Fra:0 Mem:56.10M (Peak 69.36M) | Time:00:21.13 | Compositing Fra:0 Mem:56.10M (Peak 69.36M) | Time:00:21.13 | Compositing | Determining resolution Fra:0 Mem:56.10M (Peak 69.36M) | Time:00:21.13 | Compositing | Initializing execution Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.83 | Compositing | Tile 1-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.83 | Compositing | Tile 2-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.83 | Compositing | Tile 3-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.83 | Compositing | Tile 4-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.84 | Compositing | Tile 5-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.84 | Compositing | Tile 6-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.84 | Compositing | Tile 7-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.85 | Compositing | Tile 8-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.85 | Compositing | Tile 9-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.85 | Compositing | Tile 10-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.85 | Compositing | Tile 11-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.86 | Compositing | Tile 12-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.86 | Compositing | Tile 13-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.86 | Compositing | Tile 14-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.86 | Compositing | Tile 15-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.87 | Compositing | Tile 1-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.87 | Compositing | Tile 2-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.88 | Compositing | Tile 3-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.88 | Compositing | Tile 4-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.88 | Compositing | Tile 5-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.89 | Compositing | Tile 6-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.89 | Compositing | Tile 7-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.89 | Compositing | Tile 8-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.90 | Compositing | Tile 9-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.90 | Compositing | Tile 10-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.90 | Compositing | Tile 11-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.91 | Compositing | Tile 12-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.91 | Compositing | Tile 13-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.91 | Compositing | Tile 14-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.91 | Compositing | Tile 15-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.94 | Compositing | Tile 1-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.96 | Compositing | Tile 2-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.98 | Compositing | Tile 3-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:24.99 | Compositing | Tile 4-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:25.01 | Compositing | Tile 5-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:25.03 | Compositing | Tile 6-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:25.06 | Compositing | Tile 7-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:25.08 | Compositing | Tile 8-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:25.09 | Compositing | Tile 9-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:25.12 | Compositing | Tile 10-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:25.13 | Compositing | Tile 11-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:25.17 | Compositing | Tile 12-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:25.18 | Compositing | Tile 13-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:25.20 | Compositing | Tile 14-15 Fra:0 Mem:158.12M (Peak 158.12M) | Time:00:25.22 | Compositing | Tile 15-15 Fra:0 Mem:158.05M (Peak 158.12M) | Time:00:25.23 | Compositing | De-initializing execution Saved: C:\Users\YOGESH~1\AppData\Local\Temp\blender_proc_81324f8d050e499b8d041559319a78b4\distance_0000.exr Saved: C:\Users\YOGESH~1\AppData\Local\Temp\blender_proc_81324f8d050e499b8d041559319a78b4\normals_0000.exr Fra:0 Mem:73.68M (Peak 158.12M) | Time:00:25.41 | Sce: Scene Ve:0 Fa:0 La:0 Saved: 'C:\Users\YOGESH~1\AppData\Local\Temp\blender_proc_81324f8d050e499b8d041559319a78b4\rgb_0000.png' Time: 00:26.71 (Saving: 00:01.30)
Merging data for frame 0 into J:\SaarlandUniversity\MasterThesis\BlenderProc\BlenderProc\examples/basic/output\0.hdf5 Key: colors - shape: (720, 1280, 3) - dtype: uint8 - path: C:\Users\YOGESH~1\AppData\Local\Temp\blender_proc_81324f8d050e499b8d041559319a78b4\rgb_0000.png Key: distance - shape: (720, 1280) - dtype: float64 - path: C:\Users\YOGESH~1\AppData\Local\Temp\blender_proc_81324f8d050e499b8d041559319a78b4\distance_0000.exr Key: normals - shape: (720, 1280, 3) - dtype: float64 - path: C:\Users\YOGESH~1\AppData\Local\Temp\blender_proc_81324f8d050e499b8d041559319a78b4\normals_0000.exr
Blender quit_
So as BlenderProc currently does not know how to apply these link poses, one way would be to write a custom Manipulator that loads the link pose information using
and then sets these as poses of the visual/inertial meshes.
You could orient yourself at the Entity Manipulator: https://github.com/DLR-RM/BlenderProc/blob/master/src/manipulators/EntityManipulator.py https://github.com/DLR-RM/BlenderProc/tree/master/examples/entity_manipulation
Hey @b-yogesh,
regarding your loading problem:
thanks for the explanation, I missed the info that you save your loaded robot to a .blend file and then import it via the BlendLoader.
At the moment we are appending the blend file item-by-item which seems to destroy connections between them and which in the end leads to the results you got.
As a temporary workaround you could replace the run
function in the BlendLoader (src/loader/BlendLoader.py) with the following code:
def run(self):
# get a path to a .blend file
path = Utility.resolve_path(self.config.get_string("path"))
with bpy.data.libraries.load(path) as (data_from, data_to):
data_to.collections = [name for name in data_from.collections]
for collection in data_to.collections:
bpy.context.scene.collection.children.link(collection)
This should now load the whole content of your blend file without destroying any links between them. If you then want to further manipulate specific links after loading, you can take a look at what Martin wrote above.
Thank you very much. It worked like a charm! 😄
I just have a few more follow-up questions and would appreciate if you give me some advise.
1) How do I rotate the arms using python at random different angles for each joint (I tried extracting bpy.data.objects[obj_id] and then using the rotate one joint of the arm by some degrees in some axis, but this does not respect the constraints added via URDF. Although, while using the Rotate tool, the constraints are respected). Am I missing something here (i was told that entity manipulator cannot work with robots yet)?
2) To generate a dataset, do I need to write a loop for run.py ?
3) The output of BlenderProc is a HDF file, can I change the script so that it saves only the images (rgb, seg, depth)?
Thank you
Hey,
these are great questions, but to make the issues better searchable, please do open new issues for your questions 2 and 3.
Best, Max
Sure, will do!
@b-yogesh Regarding your first question:
I think you need to change the rotation of the respective bone to make use of its constraints. Changing the rotation should look like bpy.data.objects['Back_x_Link'].pose.bones["Bone"].rotation_euler[1] = 0.1
.
With Back_x_Link
being the name of your armature object and Bone
being the name of the bone you want to manipulate.
The rotation you set should now be clipped to the configured limit interval.
You can simply generate a animation in blender (frame by frame) and I hope you can achieve the random effect. Or you can simply use object pose and manipulate the arms of a robot. My recommendation, creating a animation inside blender.
Thank you for your prompt replies and help. I got it working with this bpy.data.objects['Back_x_Link'].pose.bones["Bone"].rotation_euler[1] = 0.1
.
@b-yogesh Hi, I am trying to import the URDF file with Phobos. But I cannot import the urdf file since it throw some errors when I used the Phobos add-on. Known from your reply that you had a few changes in the script of Phobos. Would you please share more details about the revised part of the Phobos to be compatibility with Blender 2.90?
@b-yogesh Hi, I am trying to import the URDF file with Phobos. But I cannot import the urdf file since it throw some errors when I used the Phobos add-on. Known from your reply that you had a few changes in the script of Phobos. Would you please share more details about the revised part of the Phobos to be compatibility with Blender 2.90?
Hi @hetolin, sorry for the late reply. Yes, so I had used this code from the Phobos older releases for defLayers
def defLayers(layerlist):
"""Returns a list of 20 elements encoding the visible layers according to layerlist
Args:
layerlist:
Returns:
"""
if not isinstance(layerlist, list):
layerlist = [layerlist]
layers = 20 * [False]
for layer in layerlist:
layers[layer] = True
return layers
Paste this code in the following directory phobos/utils/blender.py
. Remove the installation.conf
and run the phobos setup again.
Hope this helps
@b-yogesh hi, thank you for your detailed reply. But after following the instructions as you provided. I still met a problem. The blender throws the error: `Python: Traceback (most recent call last): File "/home/hetolin/.config/blender/2.91/scripts/addons/phobos/operators/io.py", line 283, in execute models.buildModelFromDictionary(model) File "/home/hetolin/.config/blender/2.91/scripts/addons/phobos/model/models.py", line 1208, in buildModelFromDictionary model['links'][lnk]['object'] = linkmodel.createLink(link) File "/home/hetolin/.config/blender/2.91/scripts/addons/phobos/model/links.py", line 125, in createLink geometrymodel.createGeometry(vis, 'visual', newlink) File "/home/hetolin/.config/blender/2.91/scripts/addons/phobos/model/geometries.py", line 150, in createGeometry bpy.context.scene.layers = bUtils.defLayers(defs.layerTypes[geomsrc]) AttributeError: 'Scene' object has no attribute 'layers'
location:
Is the bpy.context.scene.layers
no longer used in Blender 2.9?
Hi @hetolin, can you try commenting this line
bpy.context.window.view_layer = bUtils.defLayers(defs.layerTypes[geomsrc]) line (150)
in
/home/hetolin/.config/blender/2.91/scripts/addons/phobos/model/geometries.py
@b-yogesh I tried your instruction, and it also throws the error: (I am using Blender 2.91, Maybe I need to change to Blender2.90? Have you meet such a problem when you are using phobos in Blender?)
Python: Traceback (most recent call last):
File "/home/hetolin/.config/blender/2.91/scripts/addons/phobos/operators/io.py", line 283, in execute
models.buildModelFromDictionary(model)
File "/home/hetolin/.config/blender/2.91/scripts/addons/phobos/model/models.py", line 1208, in buildModelFromDictionary
model['links'][lnk]['object'] = linkmodel.createLink(link)
File "/home/hetolin/.config/blender/2.91/scripts/addons/phobos/model/links.py", line 125, in createLink
geometrymodel.createGeometry(vis, 'visual', newlink)
File "/home/hetolin/.config/blender/2.91/scripts/addons/phobos/model/geometries.py", line 151, in createGeometry
bpy.context.window.view_layer = bUtils.defLayers(defs.layerTypes[geomsrc])
TypeError: bpy_struct: item.attr = val: Window.view_layer expected a ViewLayer type, not list
location:
Hello @hetolin, even I am using blender 2.91, and it works just fine. Although, I would like to ask, which branch of phobos have you installed? You should install this branch https://github.com/dfki-ric/phobos/tree/blender2.9
and not the master
branch
Also, I did not see this line of code
bpy.context.window.view_layer = bUtils.defLayers(defs.layerTypes[geomsrc]) Line 151
in the geometries.py
. I think you might have added it by yourself and forgot to remove it. Try removing that, and restart blender. Let me know how that goes
Hello, @b-yogesh I have installed the blender2.9 branch of phobos. And I comment bpy.context.window.view_layer = bUtils.defLayers(defs.layerTypes[geomsrc]) Line 151
. It throws new errors:
Python: Traceback (most recent call last): File "/home/hetolin/.config/blender/2.91/scripts/addons/phobos/operators/io.py", line 283, in execute models.buildModelFromDictionary(model) File "/home/hetolin/.config/blender/2.91/scripts/addons/phobos/model/models.py", line 1208, in buildModelFromDictionary model['links'][lnk]['object'] = linkmodel.createLink(link) File "/home/hetolin/.config/blender/2.91/scripts/addons/phobos/model/links.py", line 127, in createLink geometrymodel.createGeometry(col, 'collision', newlink) File "/home/hetolin/.config/blender/2.91/scripts/addons/phobos/model/geometries.py", line 204, in createGeometry newgeom.select = True AttributeError: 'Object' object has no attribute 'select'
location:
and I comment newgeom.select = True(line 204)
it seems works now.
Good to hear that it works now :). So, in blender, in the scene collection, you have one collection called 'Collision' or 'Collision joints'. I would recommend disabling that (click on the eye symbol). For coloring, you can add material to each joint. Let me know how it goes.
Thank you for your guidance. After disabling the 'Collision' collection. It works now. thank you for your detailed instructions
Glad that it worked out. :)
@b-yogesh hello, I have successfully imported the urdf file in the blender. But the 'visual' model shown in the blender(top row picture)is not visually consistent with that shown in the Rviz(bottom row picture). Noted that I used the same urdf file. Have you ever had such a problem?
Hey,
we now have URDF support in BlenderProc ;)
Best, Max
Hello,
So, I am trying to create a synthetic dataset for Robot pose estimation, and I have a URDF file. But blender does not support importing of URDF files. A plugin called Phobos (https://github.com/dfki-ric/phobos) gives blender the ability to do that, but it can only do it for Blender version 2.79, which I think BlenderProc does not support. If it is then imported, how do I control the robot using BlenderProc (entity manipulator)?
How can I approach this problem and what could be the possible solution?