search

MCG-NJU / SADRNet

[TIP 2021] SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction
Apache License 2.0
127 stars 27 forks source link

prepare_dataset.py: What is Extra_LP? train_blocks is empty #10

Closed lhyfst closed 2 years ago

lhyfst commented 3 years ago

When I run src/run/prepare_dataset.py, I encounter an error: FileNotFoundError: [Errno 2] No such file or directory: 'data/dataset/Extra_LP/all_image_data.pkl'

skip  data/dataset/300W_LP_crop/HELEN_Flip/HELEN_173153923_2_12
skip  data/dataset/300W_LP_crop/HELEN_Flip/HELEN_2236814888_2_3
skip  data/dataset/300W_LP_crop/IBUG_Flip/IBUG_image_018_5
skip  data/dataset/300W_LP_crop/LFPW/LFPW_image_train_0328_6
skip  data/dataset/300W_LP_crop/LFPW_Flip/LFPW_image_train_0380_3
skip  data/dataset/300W_LP_crop/landmarks/AFW
skip  data/dataset/300W_LP_crop/landmarks/HELEN
skip  data/dataset/300W_LP_crop/landmarks/IBUG
skip  data/dataset/300W_LP_crop/landmarks/LFPW
0 data added
saving data path list
data path list saved
0 data added
saving data path list
Traceback (most recent call last):
  File "src/run/prepare_dataset.py", line 19, in <module>
    train_dataset = make_dataset(TRAIN_DIR, 'train')
  File "./src/dataset/dataloader.py", line 217, in make_dataset
    raw_dataset.add_image_data(folder, mode)
  File "./src/dataset/dataloader.py", line 116, in add_image_data
    self.save_image_data_paths(all_data, data_dir)
  File "./src/dataset/dataloader.py", line 120, in save_image_data_paths
    ft = open(f'{data_dir}/all_image_data.pkl', 'wb')
FileNotFoundError: [Errno 2] No such file or directory: 'data/dataset/Extra_LP/all_image_data.pkl'
worker: 7 end 236/250  data/packs/AFLW2000/image00451.jpgLEN_2208472833_2_8.jpgpgg
worker: 3 end 230/15307  data/packs/300W_LP/HELEN/HELEN_2618147986_1_3.jpg.jpgpgpg
worker: 5 end 236/15307  data/packs/300W_LP/LFPW/LFPW_image_train_0304_13.jpgjpgg
worker: 1 end 242/15307  data/packs/300W_LP/IBUG_Flip/IBUG_image_030_3.jpgpg0.jpgg
worker: 4 end 245/250  data/packs/AFLW2000/image02453.jpgW_image_train_0731_3.jpg
worker: 0 end 239/15307  data/packs/300W_LP/HELEN/HELEN_248684423_1_12.jpg1_10.jpg
worker: 6 end 235/15301  data/packs/300W_LP/HELEN_Flip/HELEN_2419679570_1_5.jpg
worker: 2 end 245/15307  data/packs/300W_LP/HELEN/HELEN_2345048760_1_10.jpggjpgpg
worker: 6 end 15168/15307  data/packs/300W_LP/LFPW/LFPW_image_train_0476_15.jpgpgggg
worker: 0 end 15281/15301  data/packs/300W_LP/HELEN_Flip/HELEN_2466594504_1_9.jpgggg
worker: 5 end 15270/15307  data/packs/300W_LP/HELEN/HELEN_2882149940_1_6.jpg0.jpgg
worker: 1 end 15287/15301  data/packs/300W_LP/HELEN_Flip/HELEN_3026147764_1_11.jpg
worker: 7 end 15272/15307  data/packs/300W_LP/HELEN/HELEN_111835766_1_16.jpgjpgpgg
worker: 2 end 15259/15307  data/packs/300W_LP/LFPW/LFPW_image_train_0741_0.jpgg
worker: 3 end 15270/15307  data/packs/300W_LP/LFPW/LFPW_image_train_0051_4.jpgg
worker: 4 end 15306/15307  data/packs/300W_LP/LFPW/LFPW_image_train_0382_5.jpgg

My question is what is Extra_LP? Is it a dataset? It is not mentioned in the readme.md nor the paper. And where should I get the Extra_LP/all_image_data.pkl ?

Another problem is after I run src/run/prepare_dataset.py, I find SADRNet/data/dataset/train_blocks is generated, but it is empty. Is it correct? Or I missed something?

arukumattsun commented 3 years ago

Another problem is after I run src/run/prepare_dataset.py, I find SADRNet/data/dataset/train_blocks is generated, but it is empty. Is it correct? Or I missed something?

I was able to generate train_blocks by following these steps.

  1. comment out 12-17. https://github.com/MCG-NJU/SADRNet/blob/a5e6fac904c66711a6e9457fc603bd7b6d348d21/src/run/prepare_dataset.py#L12-L17

  2. run and delete all_image_data.pkl in 300W_LP_crop and AFLW2000_crop.(Because the pkl file created by the multi_process is not correct)

  3. Undo Step 1 and comment out 7-9. https://github.com/MCG-NJU/SADRNet/blob/a5e6fac904c66711a6e9457fc603bd7b6d348d21/src/run/prepare_dataset.py#L7-L9

  4. Change multiSaveBlock function to add all_data as an argument.multiSaveBlock(0, NUM_BLOCKS, all_data) https://github.com/MCG-NJU/SADRNet/blob/a5e6fac904c66711a6e9457fc603bd7b6d348d21/src/run/prepare_dataset.py#L17

  5. run. (By deleting pkl, the correct pkl will be generated again here. https://github.com/MCG-NJU/SADRNet/blob/a5e6fac904c66711a6e9457fc603bd7b6d348d21/src/dataset/dataloader.py#L87-L101)

arukumattsun commented 3 years ago

My question is what is Extra_LP? Is it a dataset? It is not mentioned in the readme.md nor the paper. And where should I get the Extra_LP/all_image_data.pkl ?

I'm wondering about this too.

reshow commented 3 years ago

ExtraLP is the dataset of 5,000 generated samples mentioned in Implementation Details in the paper. I'm really sorry that I can only find an early version of the code https://github.com/reshow/SDRN/blob/master/createdata.py but it doesn't match the current repo. To run this repo correctly, you can simply remove ExtraLP from the config file config_SADRN_v2.py.