search

arnoweng / CheXNet

A pytorch reimplementation of CheXNet
577 stars 221 forks source link

Running ChexNet on normal chest x ray produces FALSE positive (pretty much every time) #11

Closed taewookim closed 6 years ago

taewookim commented 6 years ago

False positives is everywhere.

I ran on few, such as this one: https://radiopaedia.org/cases/normal-chest-x-ray

Results:

[ { "Atelectasis":0.513898491859436 }, { "Cardiomegaly":0.4632066786289215 }, { "Effusion":0.7996526956558228 }, { "Infiltration":0.3642980754375458 }, { "Mass":0.4949002265930176 }, { "Nodule":0.2438926249742508 }, { "Pneumonia":0.35062509775161743 }, { "Pneumothorax":0.2875746786594391 }, { "Consolidation":0.5513240098953247 }, { "Edema":0.4362095892429352 }, { "Emphysema":0.5255514979362488 }, { "Fibrosis":0.4974825382232666 }, { "Pleural_Thickening":0.38915345072746277 }, { "Hernia":0.5669712424278259 } ]

When it fact, this is a clean Xray

I ran on few other actual CXR with no known issues.. yet its' producing all kinds of false positives.

Has anyone else noticed that the model is faulty?

arnoweng commented 6 years ago

@taewookim The model was trained on frontal X-ray images, did you test it on other types of X-ray images such as lateral ones?