Model pred seems bad when trying to fit on only one sample

[Wang-Yu-Qing]

Hi Nikolas.

I'm learning this repo and because of the limited computation resource I have, I only test the keypoints detection model upon one input sample, which is 00000.jpg. I want to see if the model can fit this single sample well. So I made the dataloader only read this single sample to train the model.

And after around 80 epochs, I saved the prediction masks with adaptive wing loss of value 0.01495 and compared it with the label masks. It seems that the result is bad:

The below code produce the raw image and the label masks:

import cv2
import numpy as np
import matplotlib.pyplot as plt

def show_heatmap(img, heatmaps):
  # accumulate all masks apart from empty class
  acc_heatmap = heatmaps[:-1, :, :].max(axis=0)
  # heatmap for empty class
  empty_class_heatmap = heatmaps[-1, :, :]

  fig, ax = plt.subplots(nrows=1, ncols=3)
  ax[0].imshow(img, cmap="gray")
  ax[1].imshow(acc_heatmap, cmap="gray")
  ax[2].imshow(empty_class_heatmap, cmap="gray")

  plt.show()

img = cv2.imread('../dataset/calibration-2023/train_v2/00000.jpg') # BGR
# this is the label masks I saved during training
hm = np.load('model_ckpt/heatmap.npy')[0]
show_heatmap(img, hm)

And the below code is to produce the pred masks:

img = cv2.imread('../dataset/calibration-2023/train_v2/00000.jpg') # BGR
# this is the masks preded by the trained model saved during training 
pred = np.load('model_ckpt/pred.npy')[0]
show_heatmap(img, pred)

It seems that the result is not good even with this one single sample. Is a awing loss of 0.01495 poor or am I missing some important things? Thanks!

[NikolasEnt]

Hi @Wang-Yu-Qing . I've conducted some experiments and can confirm the parameters don't allow good convergence on small subsets of data. As the result of my experiments, one needs at least ~1/4 of the train dataset to obtain a reasonable model.

[Wang-Yu-Qing]

@NikolasEnt You are right. I can produce a good result now.