no-mask precision is too low

[XiaoPengZong]

Hi, I have traind the dataset with your method. However the precision of no-mask class is very low,

Epoch 1000/1000
=========================

Validation cost: 0.000050
Mean average_precision (in %): 52.9850

class name      average precision (in %)
------------  --------------------------
mask                             85.9371
no-mask                          20.0329

No matter I set how many epoch of training, just like 240, 500 ,1000, the no-mast precision is too low. I also raise the size of training dataset, the result is same. Can you give me some useful advice to achieve a better result? Thanks!

[ak-nv]

We saw similar problem previously, Can you make sure if you are having right (FDDB) dataset images for no-mask once you convert to Kitti format? Since WiderFace has variety of perspectives for faces, we use complete FDDB dataset and partial WiderFace dataset. Can you add output of data2kitti.py in comments?

[XiaoPengZong]

hi, this is my command and output.

root@7a22f9e7d861:/workspace/tlt-experiments/face-mask-detection# python data2kitti.py --kaggle-dataset-path=data/Kaggle/medical-masks-dataset --mafa-dataset-path=data/MAFA --fddb-dataset-path=data/FDDB --widerface-dataset-path=data/Wider --kitti-base-path=data/kitti/ --train --category-limit=20000
Directory Already Exists
Directory Already Exists
Kaggle Dataset: Total Mask faces: 4154 and No-Mask faces:790
Total Mask Labelled:4154 and No-Mask Labelled:790
Directory Already Exists
Directory Already Exists
/workspace/tlt-experiments/face-mask-detection/data_utils/mafa2kitti.py:51: RuntimeWarning: overflow encountered in ubyte_scalars
  bbox = [_bbox_label[0], _bbox_label[1], _bbox_label[0]+_bbox_label[2], _bbox_label[1]+_bbox_label[3]]
/usr/local/lib/python3.6/dist-packages/PIL/TiffImagePlugin.py:764: UserWarning: Corrupt EXIF data.  Expecting to read 2 bytes but only got 0. 
  warnings.warn(str(msg))
/usr/local/lib/python3.6/dist-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 6553600 bytes but only got 0. Skipping tag 0
  " Skipping tag %s" % (size, len(data), tag))
/usr/local/lib/python3.6/dist-packages/PIL/TiffImagePlugin.py:747: UserWarning: Possibly corrupt EXIF data.  Expecting to read 655924 bytes but only got 65. Skipping tag 0
  " Skipping tag %s" % (size, len(data), tag))
/usr/local/lib/python3.6/dist-packages/PIL/TiffImagePlugin.py:764: UserWarning: Corrupt EXIF data.  Expecting to read 12 bytes but only got 8. 
  warnings.warn(str(msg))
MAFA Dataset: Total Mask faces: 15846 and No-Mask faces:232
Total Mask Labelled:20000 and No-Mask Labelled:1022
Directory Already Exists
Directory Already Exists
FDDB Dataset: Mask Labelled:0 and No-Mask Labelled:0
Total Mask Labelled:20000 and No-Mask Labelled:1022
WideFace: Total Mask Labelled:0 and No-Mask Labelled:18957
----------------------------
Final: Total Mask Labelled:20000
Total No-Mask Labelled:19979
----------------------------

[ak-nv]

From your log, it shows that, it does not generate FDDB dataset. Since most of data is from Widerface it has lower accuracy

I found it is a bug and I have fixed it. Please do git pull It should generate following log now, which will consist of fddb data as well

Kaggle Dataset: Total Mask faces: 4154 and No-Mask faces:790
Total Mask Labelled:4154 and No-Mask Labelled:790
Directory Already Exists
Directory Already Exists
/home/nvidia/face-mask-detection/data_utils/mafa2kitti.py:51: RuntimeWarning: overflow encountered in ubyte_scalars
  bbox = [_bbox_label[0], _bbox_label[1], _bbox_label[0]+_bbox_label[2], _bbox_label[1]+_bbox_label[3]]
MAFA Dataset: Total Mask faces: 1846 and No-Mask faces:232
Total Mask Labelled:6000 and No-Mask Labelled:1022
Directory Already Exists
Directory Already Exists
FDDB Dataset: Mask Labelled:0 and No-Mask Labelled:2845
Total Mask Labelled:6000 and No-Mask Labelled:3867
WideFace: Total Mask Labelled:0 and No-Mask Labelled:2134
----------------------------
Final: Total Mask Labelled:6000
Total No-Mask Labelled:6001
----------------------------

Please let me know if the problem still exists.

[XiaoPengZong]

Hi, your advice is useful, I retrain and get the result of below:

Epoch 500/500
=========================

Validation cost: 0.000105
Mean average_precision (in %): 83.1886

class name      average precision (in %)
------------  --------------------------
mask                             87.2816
no-mask                          79.0957

Median Inference Time: 0.006371

I will do more test for this project, thanks for your work!