lindawangg / COVID-Net

COVID-Net Open Source Initiative

Other

1.15k stars 480 forks source link

When MERS CXR fed into the CXR4-A model it classifies it as COVID 100%. Not Pneumonia and does this to other images as well. Is it a bug? #121

Closed heyeolasad closed 3 years ago

heyeolasad commented 3 years ago

Issue Template

Before posting, have you looked at the FAQ page?

Description

Please include a summary of the issue. Please include the steps to reproduce. List any additional libraries that are affected.

Steps to Reproduce

First step
Second step
Third step

Expected behavior

A description of what you expected to happen.

Actual behavior

A description of what happens instead.

Environment

Build: [e.g. 3180 - type "About" in the Command Palette]
Operating system and version: [e.g. macOS 10.14, Windows 10, Ubuntu 18.04]
[Linux] Desktop Environment and/or Window Manager: [e.g. Gnome, LXDE, i3]

haydengunraj commented 3 years ago

This result is most likely not a bug and is probably due to two main factors:

While our models are trained to distinguish COVID-19 cases from non-COVID-19 pneumonia cases, as far as I know there are not any MERS images in our datasets.
Like COVID-19, MERS is caused by a coronavirus. Some studies from last year indicate that COVID-19 shares many imaging features with MERS.

So, my best guess would be that the MERS images you tested are much more similar to COVID-19 images than other pneumonia images, and as such the network classifies them as COVID-19.