Which model is the best one

[khanhnt]

Thank a lot for your work. I tried to test your pretrained models, but I have been confused, because there are too many pretrained ones. According to your results on the validation set, the best one is FishNet150, and then MobileNet v2, MobileLite and FeatherNetA, B.

For the pretrained model of FishNet150, I still could not download from the baidu link. I have just tested with the others with 2d webcam. It seems "checkpoints/mobilenetv2_bs32/_4_best.pth.tar" model produces the most stable results. For the others (moilenetv2 which load checkpoints/pre-trainedModels/mobilenet_v2.pth.tar by default, checkpoints/FeatherNetB_bs32/_47_best.pth.tar, and checkpoints/FeatherNetA_bs32/_50_best.pth.tar), the results are alway 0. Or may be I did something wrong with this code. Here is the code I tested. Could you help me to verify the issue.

model_moilenetv2 = models.moilenetv2() #load checkpoints/pre- 
 trainedModels/mobilenet_v2.pth.tar by default

 model_MobileNetV2 = models.MobileNetV2()
 checkpoint_2 = torch.load('checkpoints/mobilenetv2_bs32/_4_best.pth.tar', map_location='cpu')
 state_dict_2 = load_weight(model_MobileNetV2,checkpoint_2)
 model_MobileNetV2.load_state_dict(state_dict_2)

 model_FeatherNetB = models.FeatherNetB()
 checkpoint_3 = torch.load('checkpoints/FeatherNetB_bs32/_47_best.pth.tar', map_location='cpu')
 state_dict_3 = load_weight(model_FeatherNetB,checkpoint_3)
 model_FeatherNetB.load_state_dict(state_dict_3)

 model_FeatherNetA = models.FeatherNetA()
 checkpoint_4 = torch.load('checkpoints/FeatherNetA_bs32/_50_best.pth.tar', 
 map_location='cpu')
 state_dict_4 = load_weight(model_FeatherNetA,checkpoint_4)
 model_FeatherNetA.load_state_dict(state_dict_4)

 if torch.cuda.is_available():
        cudnn.benchmark = True
        device = torch.device('cuda:0')
 else:
        device = torch.device('cpu')

 model_moilenetv2.eval()
 model_MobileNetV2.eval()
 model_FeatherNetB.eval()
 model_FeatherNetA.eval()

 img_size = 224
 ratio = 224.0 / float(img_size)
 normalize = transforms.Normalize(mean=[0.14300402, 0.1434545, 0.14277956],
                                 std=[0.10050353, 0.100842826, 0.10034215])
 transform = transforms.Compose([transforms.Resize(int(224 * ratio)), 
 transforms.CenterCrop(img_size),
                                transforms.ToTensor(), normalize])

 video_capture = cv2.VideoCapture(0)
 process_this_frame = True
while True:
        ret, frame = video_capture.read()
        small_frame = cv2.resize(frame, (224,224))
        process_this_frame = True
        if process_this_frame:
                pil_im = Image.fromarray(small_frame)
                img = transform(pil_im)
                img = np.array(img)
                img = np.expand_dims(img, 0)

                input = torch.tensor(img, dtype=torch.float32, device=device)

                output_1 = model_moilenetv2(input)
                output_2 = model_MobileNetV2(input)
                output_3 = model_FeatherNetB(input)
                output_4 = model_FeatherNetA(input)

                soft_output = torch.softmax(output_1, dim=-1)
                preds = soft_output.to('cpu').detach().numpy()
                _,predicted = torch.max(soft_output.data, 1)
                predicted_1 = predicted.to('cpu').detach().numpy()

                soft_output = torch.softmax(output_2, dim=-1)
                preds = soft_output.to('cpu').detach().numpy()
                _, predicted = torch.max(soft_output.data, 1)
                predicted_2 = predicted.to('cpu').detach().numpy()

                soft_output = torch.softmax(output_3, dim=-1)
                preds = soft_output.to('cpu').detach().numpy()
                _, predicted = torch.max(soft_output.data, 1)
                predicted_3 = predicted.to('cpu').detach().numpy()

                soft_output = torch.softmax(output_4, dim=-1)
                preds = soft_output.to('cpu').detach().numpy()
                _, predicted = torch.max(soft_output.data, 1)
                predicted_4 = predicted.to('cpu').detach().numpy()

                print(predicted_1,predicted_2,predicted_3,predicted_4)
                process_this_frame = False
        cv2.imshow("",frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

[maithstartup]

@khanhnt you are using RGB image to evaluate the code, whats the accuracy you are getting? my question is-(the repo is trained on depth image from chalearn dataset, does it perform well with just RGB image?

[khanhnt]

It does not perform well with each individual model, but combining three model (MobileNetV2, FeatherB and A) at once, it gives a good accuracy.

[maithstartup]

@khanhnt thanks for the answer,one last question one what type of image did you test these ensembled models with RGB image only or rgd+depth image.

[tomriddle54]

@khanhnt @maithstartup can you share your full code import libs are missing I am getting this error NameError: name 'load_weight' is not defined I think you are using Keras I would be great if someone shares full demo code to test on 1 images

[Zenvi]

your input frame is too big to contain only face. This is a face anti-spoofing project but doesn't contain a face detector. Crop your frame to a smaller size so that the frame is mostly occupied by your face. Observe the outcome then, it should be different

[baiyuang]

It does not perform well with each individual model, but combining three model (MobileNetV2, FeatherB and A) at once, it gives a good accuracy.

how to combining three model (MobileNetV2, FeatherB and A) at once ? do you mean that x1 scoreA + x2 scoreB + x3 * scoreM, but how to get the x1, x2, x3 values?