search

yuhuan-wu / JCS

[IEEE TIP 2021] COVID-CS Dataset and Code of JCS: An explainable COVID-19 diagnosis system by joint classification and segmentation
48 stars 13 forks source link

Why do you perform both AM and Segmentation? #3

Closed Niranjan008 closed 2 years ago

Niranjan008 commented 2 years ago

In your paper, why perform both Activation mapping and segmentation? What is the use of activation mapping if you are able to obtain the segmented lesions directly? Thank you.

yuhuan-wu commented 2 years ago

Thanks for your question.

1) We propose a complete diagnostic system for COVID-19. First we perform classification to know whether the person is infected with COVID-19 or not. If yes, we will perform fine-grained segmentation to know the detailed statistics for the illness. The two-step diagnosis is for saving unnecessary computational cost (perform fine-grained segmentation on healthy people).

2) The use of activation mapping is for explaining and checking that the black box CNNs learn the correct knowledge (the exact infections). In fact, black box CNNs can easily over-fit the abnormal area. It can be the infection, though. However, in our practice, CNNs can easily learn the unseen irregular patterns (brought by the scanners) and such patterns are not infections (you can refer to the visualizations of the activation mapping of different settings in our paper).