search

KeyuWu-CS / MonoHair

Code of MonoHair: High-Fidelity Hair Modeling from a Monocular Video
Other
62 stars 3 forks source link

Missing data/big_wavy1/ours/ Folder in data_processed Download Package #2

Open 0mil opened 2 weeks ago

0mil commented 2 weeks ago

Thank you for sharing this awesome project. However, I think there might be something missing to make it run correctly. I already downloaded the data_processed folder, but there is no data/big_wavy1/ours/ folder. Please check the following error message:

(MonoHair) me@ubuntu:/hair/MonoHair$ python PMVO.py --yaml=configs/reconstruct/big_wavy1
Run PMVO...
Process ID: 177603
setting configurations...
loading configs/reconstruct/base.yaml...
loading configs/reconstruct/big_wavy1.yaml...
* HairGenerate:
   * connect_dot_threshold: 0.8
   * connect_scalp: True
   * connect_segments: True
   * connect_threshold: 0.0025
   * connect_to_guide: None
   * dist_to_root: 6
   * generate_segments: True
   * grow_threshold: 0.85
   * out_ratio: 0.35
* PMVO:
   * conf_threshold: 0.15
   * filter_point: True
   * genrate_ori_only: None
   * infer_inner: True
   * num_sample_per_grid: 4
   * optimize: True
   * patch_size: 7
   * threshold: 0.025
   * visible_threshold: 1
* bbox_min: [-0.32, -0.32, -0.24]
* bust_to_origin: [0.006, -1.644, 0.01]
* camera_path: camera/calib_data/wky07-22/cam_params.json
* check_strands: True
* cpu: None
* data:
   * Conf_path: conf
   * Occ3D_path: ours/Occ3D.mat
   * Ori2D_path: best_ori
   * Ori3D_path: ours/Ori3D.mat
   * bust_path: Bust/bust_long.obj
   * case: big_wavy1
   * depth_path: render_depth
   * frame_interval: 7
   * image_size: [1920, 1080]
   * mask_path: hair_mask
   * raw_points_path: ours/colmap_points.obj
   * root: data
   * scalp_path: ours/scalp_tsfm.obj
   * strands_path: ours/world_str_raw.dat
* device: cuda:0
* gpu: 0
* image_camera_path: ours/cam_params.json
* infer_inner:
   * render_data: True
   * run_mvs: True
* name: 10-16
* ngp:
   * marching_cubes_density_thresh: 3.0
* output_root: output
* prepare_data:
   * fit_bust: None
   * process_bust: True
   * process_camera: True
   * process_imgs: True
   * render_depth: True
   * run_ngp: True
   * select_images: True
* save_path: refine
* scalp_diffusion: None
* seed: 0
* segment:
   * CDGNET_ckpt: assets/CDGNet/LIP_epoch_149.pth
   * MODNET_ckpt: assets/MODNet/modnet_photographic_portrait_matting.ckpt
   * scene_path: None
* vsize: 0.005
* yaml: configs/reconstruct/big_wavy1
existing options file found (identical)
unable to load materials from: ./bust_long_c.obj.mtl
[Open3D WARNING] Unable to load file data/big_wavy1/ours/scalp_tsfm.obj with ASSIMP
/home/users/me/miniconda3/envs/MonoHair/lib/python3.10/site-packages/numpy/core/fromnumeric.py:3432: RuntimeWarning: Mean of empty slice.
  return _methods._mean(a, axis=axis, dtype=dtype,
/home/users/me/miniconda3/envs/MonoHair/lib/python3.10/site-packages/numpy/core/_methods.py:182: RuntimeWarning: invalid value encountered in divide
  ret = um.true_divide(
Traceback (most recent call last):
  File "/home/users/me/w/hair/MonoHair/PMVO.py", line 820, in <module>
    scalp_max = np.max(scalp_vertices,axis=0)
  File "<__array_function__ internals>", line 180, in amax
  File "/home/users/me/miniconda3/envs/MonoHair/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 2793, in amax
    return _wrapreduction(a, np.maximum, 'max', axis, None, out,
  File "/home/users/me/miniconda3/envs/MonoHair/lib/python3.10/site-packages/numpy/core/fromnumeric.py", line 86, in _wrapreduction
    return ufunc.reduce(obj, axis, dtype, out, **passkwargs)
ValueError: zero-size array to reduction operation maximum which has no identity
KeyuWu-CS commented 2 weeks ago

You must first run prepare_data.py , it will create ours folder

0mil commented 2 weeks ago

@KeyuWu-CS
Thank you for your response!

While running python prepare_data.py --yaml=configs/reconstruct/big_wavy1 step following your instructions, I discovered that the ours/Occ3D.mat files were not generated, leading to the following error when executing python infer_inner.py --yaml=configs/reconstruct/big_wavy1 step. It is correct that the ours/Occ3D.mat file should be generated after running prepare_data.py successfully, right?

Error message: The numpy.ndarray strands is empty!

(MonoHair) me@ubuntu:/home/users/me/MonoHair$ python infer_inner.py --yaml=configs/reconstruct/big_wavy1
Process ID: 200078
setting configurations...
loading configs/reconstruct/base.yaml...
loading configs/reconstruct/big_wavy1.yaml...
* HairGenerate:
   * connect_dot_threshold: 0.8
   * connect_scalp: True
   * connect_segments: True
   * connect_threshold: 0.0025
   * connect_to_guide: None
   * dist_to_root: 6
   * generate_segments: True
   * grow_threshold: 0.85
   * out_ratio: 0.35
* PMVO:
   * conf_threshold: 0.15
   * filter_point: True
   * genrate_ori_only: None
   * infer_inner: True
   * num_sample_per_grid: 4
   * optimize: True
   * patch_size: 7
   * threshold: 0.025
   * visible_threshold: 1
* bbox_min: [-0.32, -0.32, -0.24]
* bust_to_origin: [0.006, -1.644, 0.01]
* camera_path: camera/calib_data/wky07-22/cam_params.json
* check_strands: True
* cpu: None
* data:
   * Conf_path: conf
   * Occ3D_path: ours/Occ3D.mat
   * Ori2D_path: best_ori
   * Ori3D_path: ours/Ori3D.mat
   * bust_path: Bust/bust_long.obj
   * case: big_wavy1
   * depth_path: render_depth
   * frame_interval: 7
   * image_size: [1920, 1080]
   * mask_path: hair_mask
   * raw_points_path: ours/colmap_points.obj
   * root: data
   * scalp_path: ours/scalp_tsfm.obj
   * strands_path: ours/world_str_raw.dat
* device: cuda:0
* gpu: 0
* image_camera_path: ours/cam_params.json
* infer_inner:
   * render_data: True
   * run_mvs: True
* name: 10-16
* ngp:
   * marching_cubes_density_thresh: 3.0
* output_root: output
* prepare_data:
   * fit_bust: None
   * process_bust: True
   * process_camera: True
   * process_imgs: True
   * render_depth: True
   * run_ngp: True
   * select_images: True
* save_path: refine
* scalp_diffusion: None
* seed: 0
* segment:
   * CDGNET_ckpt: assets/CDGNet/LIP_epoch_149.pth
   * MODNET_ckpt: assets/MODNet/modnet_photographic_portrait_matting.ckpt
   * scene_path: None
* vsize: 0.005
* yaml: configs/reconstruct/big_wavy1
generate segments...
0it [00:00, ?it/s]
0it [00:00, ?it/s]
0it [00:00, ?it/s]
done...
Warning: 'strands' is empty. Nothing to save.
unable to load materials from: ./bust_long_c.obj.mtl
num of strands: 0
Traceback (most recent call last):
  File "/home/users/me/MonoHair/infer_inner.py", line 71, in <module>
    render_data(camera ,strands ,vertices ,faces ,[1280 ,720] ,os.path.join(args.data.root ,'imgs'))
  File "/home/users/me/MonoHair/Utils/Render_utils.py", line 272, in render_data
    renderStrands = StrandsObj(strands, Render.ctx)
  File "/home/users/me/MonoHair/Utils/Render_utils.py", line 26, in __init__
    self.Lines = np.concatenate(self.Lines, 0)
  File "<__array_function__ internals>", line 180, in concatenate
ValueError: need at least one array to concatenate
KeyuWu-CS commented 2 weeks ago

3D Ori will be created in Output/experiment_name/refine/Ori.mat after run PMVO.py. And another important thing is you must notice if there are any step has error when run prepare_data.py. There are lots of steps, if one step has problem, it will have error info in the output, but the program will not stop. So you should notice it.

KeyuWu-CS commented 2 weeks ago

the full folder of our one driven has one case now, you can check it

0mil commented 2 weeks ago

Thank you for the information. I would make sure to monitor closely each step of 3D Hair Reconstruction process and recheck for any errors in the prepare_data.py step. Thanks again for your assistance.

0mil commented 1 week ago

@KeyuWu-CS
I have one last question. Thank you in advance for your response. What conversion should be used to import Cyhair file format(.hair) generally into 3D software? I have tried using several open-source converters to view the resulting connected_strands.hair in Blender or Unreal, but they do not seem to work properly.

KeyuWu-CS commented 1 week ago

You can export to Blender and convert to particle system, then create a suitable render material to render different style hair. We will also provide a .exe program to visualize the .hair in a simple render maner in next one or two week. You also can visualize it simply with open3d.

0mil commented 5 days ago

@KeyuWu-CS Thank you for your response, and I apologize for the continued questions. I have already attempted to import the model's result file, big_wavy1/output/10-16/full/connected_strands.hair, into Blender. However, Blender doesn't support the .hair format. Is there a way to import a .hair file into Blender, perhaps by converting it to another format or by installing a plugin that supports .hair file import? Or is it possible that I am mistaken about the generated hair model file being connected_strands.hair?

The results_directory I think :

./big_wavy1_result/output/10-16/full
├── coarse.npy
├── coarse_ori.npy
├── connected_strands.hair             # result "3D hair model" I think 🤔
├── num_root.npy
├── Occ3D.mat
├── Ori3D.mat
├── scalp_segment.hair
├── scalp_segment_smooth.hair
└── strands.hair
KeyuWu-CS commented 5 days ago

@KeyuWu-CS Thank you for your response, and I apologize for the continued questions. I have already attempted to import the model's result file, big_wavy1/output/10-16/full/connected_strands.hair, into Blender. However, Blender doesn't support the .hair format. Is there a way to import a .hair file into Blender, perhaps by converting it to another format or by installing a plugin that supports .hair file import? Or is it possible that I am mistaken about the generated hair model file being connected_strands.hair?

The results_directory I think :

./big_wavy1_result/output/10-16/full
├── coarse.npy
├── coarse_ori.npy
├── connected_strands.hair             # result "3D hair model" I think 🤔
├── num_root.npy
├── Occ3D.mat
├── Ori3D.mat
├── scalp_segment.hair
├── scalp_segment_smooth.hair
└── strands.hair

Yep, "connected_strands.hair" is the finally results. Bur for visualization, Blender cannot directly use .hair file, it need some expertise to render realistic hair. It's not easy If you are not familiar with Blender. It needs to convert data formats, set up cameras, lighting materials, etc. You can ask for help from someone who is familiar with rendering hair in blender. Or just use open3D to visualize the results, this may not show its geometry well. You also can wait a few days, we will release a simple visualization tools.

0mil commented 2 days ago

@KeyuWu-CS
Oh, I see. I want to create a script to convert the .hair format file to others such as .obj, .abc, or .ply. So, is this document(http://www.cemyuksel.com/research/hairmodels/) for the hair format you used? If not, can I get a any HAIR File Format Specification you used for implementing the utilities?

KeyuWu-CS commented 1 day ago

You can find the store format of .hair in https://github.com/KeyuWu-CS/MonoHair/blob/eaadc74d0e24210658741ba49ec8d59a6f5e6a5f/Utils/Utils.py#L25