mk-minchul / AdaFace

MIT License
663 stars 122 forks source link

Error on Output_layer in custom image data training using folders #100

Closed ANDRESHZ closed 1 year ago

ANDRESHZ commented 1 year ago

@mk-minchul and @ZonePG I was trying for long time to fine tuning the model using a custom folder with 8568 images, following all steps and the issue solution and suggestions that I found, but I still had been gotten the same error on output_layer of the backbone, So I can not training because that. I am using this script

!python main.py \
    --data_root /home/benjamin/Proy/DS/SEJONG/SEJ \
    --train_data_path train \
    --val_data_path faces_emore \
    --prefix ir18_ms1mv2_adaface \
    --gpus 4 \
    --use_16bit \
    --arch ir_18 \
    --batch_size 32 \
    --num_workers 10 \
    --epochs 33 \
    --lr_milestones 12,20,24 \
    --lr 0.1 \
    --head adaface \
    --m 0.4 \
    --h 0.333 \
    --custom_num_class 62 \
    --low_res_augmentation_prob 0.2 \
    --crop_augmentation_prob 0.25 \
    --photometric_augmentation_prob 0.2 \
    --start_from_model_statedict pretrained/adaface_ir18_vgg2.ckpt

I have access to a 4 GPUs imagen

THE ERROR:

File "/home/benjamin/MOD_ADAFACE/net.py", line 328, in forward
    x = self.output_layer(x)
...
RuntimeError: mat1 and mat2 shapes cannot be multiplied (5x2984448 and 25088x512)

Please can you give some ideas or method to deal with

otsebriy commented 1 year ago

I'm curious @ANDRESHZ If you have managed to fix this error?

ANDRESHZ commented 1 year ago

@otsebriy Not yet, but if you want we can schedule a meeting to see our code, and maybe together with all the information of our trials, could find a way to run. this is my email bahuerfano@uc.cl

otsebriy commented 1 year ago

@otsebriy Not yet, but if you want we can schedule a meeting to see our code, and maybe together with all the information of our trials, could find a way to run. this is my email bahuerfano@uc.cl

In my case, the problem was in the input image shape.

ANDRESHZ commented 1 year ago

@otsebriy Not yet, but if you want we can schedule a meeting to see our code, and maybe together with all the information of our trials, could find a way to run. this is my email bahuerfano@uc.cl

In my case, the problem was in the input image shape.

@otsebriy So is the MTCNN image crop not enough or is it that some images exceeded that size? Could you give me some additional details about how youe achieve this? Anyway I am already changing the dataset so that you keep the fixed size 112x112.

ANDRESHZ commented 1 year ago

@otsebriy Not yet, but if you want we can schedule a meeting to see our code, and maybe together with all the information of our trials, could find a way to run. this is my email bahuerfano@uc.cl

In my case, the problem was in the input image shape.

@otsebriy So is the MTCNN image crop not enough or is it that some images exceeded that size? Could you give me some additional details about how youe achieve this? Anyway I am already changing the dataset so that you keep the fixed size 112x112.

That was the solution, changing the image size in the dataset