search

chungyiweng / humannerf

HumanNeRF turns a monocular video of moving people into a 360 free-viewpoint video.
MIT License
786 stars 86 forks source link

Update - Dataloader issue with custom training #70

Closed Dipankar1997161 closed 1 year ago

Dipankar1997161 commented 1 year ago

Hello @chungyiweng,

I saw the following issue regarding the data loader here https://github.com/chungyiweng/humannerf/issues/4#issue-1275272277. I have the same issue right now. From the error log there, its may not be clear, so here is a little more info.

I compared the zju mocap and monocular data using "patch_mode" and got the following error log Zju-mocap zjumocap img

Monocular - in my case Human3.6m wild patches

The 'rays', 'far', 'patch_indices' are empty in monocular cases which means apparently there might be error in the SMPL joint locations in the near and far plane which falls along the camera rays for sampling.

Any suggestions on what could be the possible reason for this and any solution advice on how this can be achieved? Looking forward to discuss on this.

My image and mask are as follows 001611 001611

Dipankar1997161 commented 1 year ago

The issue is probably with the "Joints" value in the mesh_info.pkl, becoz of which 2 things happen:

  1. Ray, Far list return empty list
  2. Bbox value range is enormous in all the 3 axis

hence when the rays are projected, the candidate_mask returns 0, thus terminating the data_loader.

I may be wrong too, Would like to hear from you on this @chungyiweng

DredLiu03 commented 4 months ago

Hi, have you solved the problem?