CASIA-Webface convert data error....

[Light--]

excuse me, where can i find the train.rec ?

$ python tools/convert_data.py -r /data2/Dataset/celebA -o /data2/user1/celebA_112x112/
terminate called after throwing an instance of 'dmlc::Error'
  what():  [11:36:32] src/io/local_filesys.cc:209: Check failed: allow_null:  LocalFileSystem::Open "/data2/Dataset/celebA/train.rec": No such file or directory
Stack trace:
  [bt] (0) /home/user1/miniconda3/lib/python3.7/site-packages/mxnet/libmxnet.so(+0x3c598b8) [0x7fdc735e08b8]

Aborted (core dumped)

[XiaohangZhan]

you should download the dataset first.

[XiaohangZhan]

Besides, celebA is not related to this repo.

[Light--]

Besides, celebA is not related to this repo.

Hi, please note that the problem i encountered is not related with celeba. Of course i have downloaded the dataset, but neither celeba nor webface have the file: train.rec Only have train.txt at most. The problem is how to convert the train.txt to train.rec that compatible with your code. I have tried conversion, but also reported error:

$ python tools/convert_data.py -r /data2/Dataset/celebA -o /data2/user1/celebA_112x112/
Traceback (most recent call last):
  File "tools/convert_data.py", line 58, in <module>
    main()
  File "tools/convert_data.py", line 54, in main
    load_mx_rec(args.rec_path, args.output_path, write_img=True)
  File "tools/convert_data.py", line 18, in load_mx_rec
    max_idx = int(header.label[0])
TypeError: 'float' object is not subscriptable

[XiaohangZhan]

It is weird that the data you download does not contain a train.rec. Did you download it following the instructions in README, i.e., from https://github.com/deepinsight/insightface/wiki/Dataset-Zoo? If so, for CASIA-Webface, the folder contains files as follows: agedb_30.bin cfp_ff.bin cplfw.bin property train.lst calfw.bin cfp_fp.bin lfw.bin train.idx train.rec

[Light--]

finally the errors no longer report... i think it was related to the dataset file preparation.

1. First you should creat a data directory in the root of your downloaded repo sudo mkdir data
2. Then create a data/webface softlink file to link 'data/webface' to '/your_dir/webface ' sudo ln -s /your_dir/webface data/webface
3. Last, place the data files as the files tree should be: ├──/your_dir/webface │   ├── agedb_30.bin │   ├── calfw.bin │   ├── cfp_ff.bin │   ├── cfp_fp.bin │   ├── cplfw.bin │   ├── images │   ├── lfw.bin │   ├── list.txt │   ├── names.txt │   ├── test.txt │   └── train.txt

[XiaohangZhan]

Thank you for reporting that. The readme has been revised accordingly.

[Light--]

Thank you for reporting that. The readme has been revised accordingly.

My pleasure.

By the way, in your paper, a multi-task fashion is used to update the model. May i ask you a question?

I am studying deep facial multi-task learning and need to build a multi-task model for trainging. Do you have any good and clean implementation of multi-task CNN architecture to recommend ? Or can you share the source/reference you used for implementing the multi-task part of your work (this repo) at that time? It would be very appreciated if the source has implementation code.

Thanks a lot.