Bounding Boxes for Lungs

[oplatek]

This proposal suggests a principal solution to build a model that will predict Bounding Box (BB) around lungs, artifacts and other well-defined objects on the images.

The COVID19 and other x-ray chest datasets contain chest X-ray images capturing not only Lungs but sometimes also arms, neck, etc. The images also contain text labels and other artifacts.

The advantages:

• If the model is trained to differentiate between artifacts and the chest ray region - it can help as a form of regularization for COVID19 detection precision
• Bounding box chest array prediction will allow us explicitly compute recall - it helps with model understanding and evaluation
  • If the image does not contain lungs or the model does not focus on lungs (e.g. it focuses on an arm, or on an artifact - textual label in the image), we know that the model classification is poor even if it predicts the COVID19 label correctly
• It would be nice to detect key regions for detecting COVID19 for doctors to review (See also disadvantages: lack of BBs)

The disadvantages:

• BB predictions is a much harder task than classification
  • Having so few data, it can prove too hard to train the model, letting the classification to fail completely
    • With more data it should not be a problem -> what is the amount of the data to start using BB?
• The annotations for Bounding Boxes are generally missing
  • Some Bounding Boxes types can be automatically created - we can easily generate artifacts and add them to chest ray images
  • We can create heuristics for selecting lungs regions

Example images with textual artifacts, "pipes in lungs artifact", the edge of image artifact, ..

[edgarriba]

the disadvantages you mention are quite heavy and probably making unfeasible to obtain an solution to the problem. This seems more an unsupervised problem and probably stuff like creating self attention maps could help.

Actually found this interesting post using it already for xray images https://towardsdatascience.com/self-attention-in-computer-vision-2782727021f6

[edgarriba]

@anguelos @ducha-aiki any experience with self-attention ?

[ducha-aiki]

For explanations I'd go for this: https://arxiv.org/abs/2001.08593

For training: https://github.com/sdoria/SimpleSelfAttention