search

comsaint / dlh_project

For CS598 Deep Learning for Healthcare project
0 stars 0 forks source link

Implement setup for multi-label classification #17

Closed comsaint closed 3 years ago

comsaint commented 3 years ago

Previously in @AsadBinImtiaz 's branch, the Dataset object generates multiple labels (e.g. [1, 0, 0, 0, 1, 1, ..., 0]) for each image, which is good for multi-label classification. Unfortunately, I overwrote the work due to time constraint but implemented setting for binary classification instead. Now it is a good time to implement the multi-label setting.

Tasks:

  1. Edit data_processing.py and dataset.py so that the data loader produces a label vector (instead of 1 label).
  2. Change the loss function to binary cross-entropy loss.
  3. Modify the training and evaluation functions in train_model.py to adapt for new loss and metric (Macro-average ROC?). Also, pay attention to the last classification layer, making sure it is not using Softmax.
  4. (Question) how to handle class weight and stratify strategy?
  5. (Question) how to implement early stopping?

Notes:

  1. Do not confuse multi-class with multi-label classification. link
  2. Good reference on loss function. link
comsaint commented 3 years ago

Items 1-3 is resolved by d919051. Q4 is still a puzzle to me - looks like there is no easy solution. Item 5 will be implemented later.

comsaint commented 3 years ago

Class weight is implemented in misc branch and will become a separate issue. Early stopping is implemented as checkpointing the model if its validation loss is better than in previous epochs.