How can we use OmniXai for object detection and Segmentation task?

[Fardeenkhan1]

I was trying to run Integrated Gradient technique for some classification models and i am able to do that. But how can we use OmniXAI for models that gives bounding box as output?

[yangwenz]

Hi, you need to implement a wrapper over the object detection model so that the output of this model wrapper is the object class or the object location. This wrapped model should have a similar interface as image classification models.

[Fardeenkhan1]

Hi, Do you have some guidelines or implementations on how to achieve this with object detection model.

[yangwenz]

We don't have specific examples yet. But you can check the implementation of GradCAM:

https://github.com/salesforce/OmniXAI/blob/main/omnixai/explainers/vision/specific/gradcam/gradcam.py https://github.com/salesforce/OmniXAI/blob/main/omnixai/explainers/vision/specific/gradcam/pytorch/gradcam.py

These two files provide enough info for how to implement a wrapper. The basic idea is that the input is an image, the output is either class label or object location which can be used to compute losses.