search

PerceivingSystems / bedlam_render

BEDLAM (CVPR 2023) render pipeline tools
https://bedlam.is.tuebingen.mpg.de/
Other
131 stars 6 forks source link

Issues and Clarifications Needed on Camera Adjustments, Lighting, and Automation in UE5.0.3 #17

Closed niuzehai closed 10 months ago

niuzehai commented 10 months ago

Dear Author,

I am currently working on a project using Unreal Engine 5.0.3 on a Windows 10 environment and have encountered several issues and areas where I need clarification. Your guidance would be greatly appreciated.

  1. Manual Adjustment of Camera in Rendering: Is it required to manually adjust the camera's translation and rotation in UE5 during the rendering step?

  2. Scene Lighting and HDRI Configuration: Does the scene lighting, including HDRI, need to be manually configured for each scene?

  3. Automation for Camera and Lighting Setup: Is there an existing method or script available for automatically generating a large number of scenes with pre-configured cameras and lighting?

  4. Accessing Camera and SMPL Parameters: How can I obtain the camera's intrinsic and extrinsic parameters, as well as the SMPL parameters for an image?

  5. Generating Multi-View Sequences: Could you provide guidance on how to generate sequences from multiple viewpoints?

  6. Multi-View Camera Calibration: How can I use a set of multi-view cameras, calibrated with a chessboard grid, to generate a dataset?

  7. Error in Reading Log Post-Rendering: Finally, I encountered an error while reading the log after rendering. The error log is as follows:

LogScript: Error: Script Msg: Traceback (most recent call last): File "D:/UE_5.0/Engine/Content/PS/Bedlam/Core/Python/render_movie_render_queue.py", line 46, in OnIndividualJobFinishedCallback export_camera_data(sequence_name) File "D:/UE_5.0/Engine/Content/PS/Bedlam/Core/Python/render_movie_render_queue.py", line 64, in export_camera_data lines = fp.readlines() UnicodeDecodeError: 'gbk' codec can't decode byte 0xad in position 2823: illegal multibyte sequence LogScript: Error: Script call stack: OnMoviePipelineIndividualJobFinished__PythonCallable.CallPython

Could you please provide some insight or a solution to this error?

Thank you for your time and assistance. Looking forward to your response.

niuzehai commented 10 months ago

Hello,

I've been delving into the issues and areas of clarification regarding the use of Unreal Engine 5.0.3 on a Windows 10 environment, and I'd like to share my findings thus far:

  1. Manual Adjustment of Camera in Rendering I discovered that the camera's translation and rotation during the rendering step in UE5 can be efficiently adjusted using be_modify_sequences.py. Additionally, manual adjustments can be made through BE_CIneCameraActor_Blueprint within UE5.

  2. Scene Lighting and HDRI Configuration For scene lighting, including HDRI, it seems feasible to configure each scene either through the be_generate_sequences_crowd.py script or by manually tweaking the BE_HDRIBackdrop in UE5.

  3. Automation for Camera and Lighting Setup Regarding the automation of camera and lighting setups, I found that be_modify_sequences.py can be employed to automate the generation of scenes with pre-configured cameras. As for lighting setup, manual adjustments to BE_HDRIBackdrop appear to be effective.

  4. Accessing Camera and SMPL Parameters This is an area I'm still exploring – specifically, how to access the camera's intrinsic and extrinsic parameters, as well as the SMPL parameters for an image.

  5. Generating Multi-View Sequences I've ascertained that generating sequences from multiple viewpoints can be accomplished by manually altering the camera positions and rendering different sequences repetitively.

  6. Multi-View Camera Calibration This aspect seems to involve reverse-engineering Unreal Engine configurations from camera parameters (both intrinsic and extrinsic), initially calibrated using a chessboard grid.

  7. Error in Reading Log Post-Rendering Finally, the error encountered while reading the log post-rendering appears to be related to an issue with the log file's encoding.

I'm eager to receive any feedback or additional insights on these findings. Thank you!

tpsmpi commented 10 months ago

2+3: If you pass the path to HDRI whitelist file to be_generate_sequences_crowd.py it will automatically set the target HDRI used per sequence in the generated be_seq.csv file. You can further randomize the view into the HDRI via be_modify_sequences.py sequenceroot. Manual tweaking is not needed for this when you use the BE_IBL reference project.

4: See #18

niuzehai commented 10 months ago

Dear tpsmpi,

I want to extend my heartfelt thanks for your prompt and insightful response to my queries regarding Unreal Engine 5.0.3. Your guidance has been incredibly helpful.

Regarding points 2 and 3, your explanation about using the HDRI whitelist file with be_generate_sequences_crowd.py to automatically set the target HDRI per sequence is precisely the solution I needed. It's reassuring to know that manual tweaking isn't necessary with the BE_IBL reference project, and that be_modify_sequences.py can be used for further randomization. This clarification will streamline my workflow significantly.

As for point 4, I have reviewed the information in issue #18, and it has provided me with the clarity I was seeking on accessing camera and SMPL parameters. Your direction here is greatly appreciated and has effectively resolved my concerns.

I am grateful for your time and assistance in addressing these aspects of my project. Your expertise has not only solved my immediate challenges but also enhanced my overall approach to working with Unreal Engine.

Thank you once again for your invaluable support.