Question regarding GT

[HannahHaensen]

I thanks for the nice plugin. works well for me. However, I wanted to use nerf just as a downstream task thus i wanted to create some more information for the output:

https://github.com/DLR-RM/BlenderProc/issues/895

do you have an idea why it is not working, with your plugin it works fine, but when getting the transformation matrix from here it just produces weird things. even when i render the same images.

[maximeraafat]

Hi @HannahHaensen,

Do I understand it correctly that you are experiencing an issue with matching the views between Blender and BlenderProc? Unfortunately I am unfamiliar with BlenderProc, so I would appreciate if you could provide a bit more details on what your issue is exactly, and which are the steps you followed in order to get there. Thanks in advance!

[HannahHaensen]

@maximeraafat thanks for your reply.

So what i do I have an object in the scene like the donut and then I sample random camera poses facing towards the object of interest in the scene.

what i experience is that with your random pose sampling it is working however, with mine i face the issue off weird blurring

this is the result with mine:

this is the result with your code

i was just wondering if it could be because of camera poses or sth.

[HannahHaensen]

if i keep training with the images provided by your plugin the result is fine, while mine stays like in the image

[maximeraafat]

Hi @HannahHaensen, your issue must either lie with the camera poses or with the camera angle. I am unsure of what your code exactly does, since some functions seem to be handled by BlenderProc. I would suggest however comparing the transform_matrix and the camera angles outputs from your code to the ones from my plugin.

If the camera matrices are not approximately the same, your camera poses are wrong, in which case I suggest simply copying my code to extract camera poses instead of using the BlenderProc built-in functions. I compute all the camera intrinsics and extrinsics in this file, feel free to copy what you need for your application!

Let me know if this helps or whether you are facing more issues :)

[HannahHaensen]

Hey @maximeraafat I did this so for the camera angle i do the following:

def get_camera_angle_x() -> float:
    """ Returns the camera angle_x as float.

    :param frame: None
    :return: angle x.
    """
    cam = bpy.data.cameras["Camera"]

    print('blender angle_x', bpy.data.objects['Camera'].data.angle_x)
    print('blender angle_y', bpy.data.objects['Camera'].data.angle_y)

    print(cam.angle_x)

    return cam.angle_x

and for the extrinsic i use the code from blenderproc, only difference i could spot compared to your code were some after the 6 point of the decimal numbers

[maximeraafat]

Hmm this is strange. If the extrinsics and camera angles are the same, then your code should produce similar results. Have you verified that all your camera matrices align with the ones that my code produces, not just for a single camera? You could write a for loop that verifies that each camera matrix from your code is the same as the ones from my code, within a certain threshold