search

lindawangg / COVID-Net

COVID-Net Open Source Initiative
Other
1.15k stars 480 forks source link

Classifier only returning one class. #102

Open palatos opened 3 years ago

palatos commented 3 years ago

I have tested two different models available on this repository, and they both return class 2 (covid) for just about any input I use, from any of the common covid or CXR pneumonia datasets being used right now. I know there is a similar issue open, but I thought I'd reinforce this and perhaps ask for the repository to have some easy way to reproduce results. On the Results section of README, do the [100 Covid] tests mean the network was tested on 100 covid images? Are we sure the network is not just outputting Covid 90% of the time regardless of the inputs?

I believe in the interest of replicating these results it would be important to test this further, or maybe explain why this problem is happening. The inference code only includes a 1./255 normalization as preprocessing. Is there any other aditional preprocessing perhaps missing?

haydengunraj commented 3 years ago

Hi, thanks for bringing this issue up. We're looking into it, but to help us out could you point us to the names/locations of the images you're using so that we can replicate the problem and investigate further?

palatos commented 3 years ago

Thank you so much for the answer. I prepared a notebook with an example of what I've been trying. https://github.com/palatos/mynotes/blob/main/weight-tests.ipynb For the images I cloned https://github.com/agchung/Actualmed-COVID-chestxray-dataset. For the weights I used the COVIDNet-CXR4-A checkpoint available in the README of this repo.

You'll notice the very high amount of covid predictions. I think I'm doing all the same steps as the original code, on that notebook.

Please let me know if I can do anything else to help.