about metric - Githubissues

Linaom1214 commented 1 month ago

great work! But when I try to reproduce this program, the metrics “I-Auroc、I-F1、I-AP” is always zero. is right?

  0%|          | 0/5 [00:00<?, ?it/s]24-07-29 15:45:27.940 - INFO: epoch [1/5], loss:0.7195
24-07-29 15:45:27.941 - INFO: =============================Testing ====================================
24-07-29 15:48:54.038 - INFO:         ColonDB       I-Auroc:0.00    I-F1:0.00   I-AP:0.00   P-Auroc:82.30   P-F1:35.36  P-AP:28.43
24-07-29 15:48:54.039 - INFO:         Average       I-Auroc:0.00    I-F1:0.00   I-AP:0.00   P-Auroc:82.30   P-F1:35.36  P-AP:28.43

 20%|██        | 1/5 [15:18<1:01:15, 918.89s/it]24-07-29 16:00:49.425 - INFO: epoch [2/5], loss:0.5814
24-07-29 16:00:49.425 - INFO: =============================Testing ====================================
24-07-29 16:04:15.806 - INFO:         ColonDB       I-Auroc:0.00    I-F1:0.00   I-AP:0.00   P-Auroc:79.62   P-F1:30.77  P-AP:23.65
24-07-29 16:04:15.806 - INFO:         Average       I-Auroc:0.00    I-F1:0.00   I-AP:0.00   P-Auroc:79.62   P-F1:30.77  P-AP:23.65

 40%|████      | 2/5 [30:40<46:01, 920.53s/it]  24-07-29 16:16:09.737 - INFO: epoch [3/5], loss:0.5844
24-07-29 16:16:09.738 - INFO: =============================Testing ====================================

caoyunkang commented 1 month ago

Hi! Thanks for your interest!

This is because all images in the ColonDB dataset are abnormal. Thus, the ColonDB dataset only supports the anomaly localization task.

Linaom1214 commented 1 month ago

Hi! Thanks for your interest!

This is because all images in the ColonDB dataset are abnormal. Thus, the ColonDB dataset only supports the anomaly localization task. @caoyunkang

thanks! In order to migrate this project to other tasks, do I need to construct data that includes both anomalies and normal data? If I only want to achieve the segmentation of anomaly targets (using only one type of segmentation data), is this not feasible?

caoyunkang commented 1 month ago

thanks! In order to migrate this project to other tasks, do I need to construct data that includes both anomalies and normal data? If I only want to achieve the segmentation of anomaly targets (using only one type of segmentation data), is this not feasible?

Hi, it depends on the aimed task. Technically, the auxiliary data utilized for training should comprise both normal and abnormal samples, with both image-level and pixel-level annotations. For the testing data, arbitrary inputs are acceptable. @Linaom1214

Linaom1214 commented 1 month ago

@caoyunkang Thank you very much for your answer. I have one more question. I want to perform validation on a single-category task. Can I achieve this by providing the following labels?

          {
              "img_path": "xx",
              "mask_path": "xx",
              "cls_name": "object",
              "specie_name": "",
              "anomaly": 0
          },
          {
              "img_path": "xx",
              "mask_path": "xx",
              "cls_name": "object",
              "specie_name": "",
              "anomaly": 1            },

caoyunkang commented 1 month ago

@Linaom1214 I am not very clear about your question. You should offer both img_path and mask_path. For the cls_name, it would be fine to simply use object.

Linaom1214 commented 1 month ago

@Linaom1214 I am not very clear about your question. You should offer both img_path and mask_path. For the cls_name, it would be fine to simply use object.

Thank you for your patient explanation, it has been very helpful to me.

caoyunkang / AdaCLIP

about metric #4