Explaining regression for images

[MTDzi]

Hello,

tl;dr I would like to use LIME for explaining regression for images.

I'm doing a self-driving car project where my model is predicting the angle of the wheels needed for the car to take a turn (for each frame independently, no history taken into account). Every once in a while, the model decides to inexplicably get off the road, and I asked myself: "Can LIME help me understand what made it do that?"

In your "Roadmap" section of the contributing guidelines, point 4., I found:

Thinking through what is needed to use LIME in regression problems. An obvious problem is that features with different scales make it really hard to interpret.

On the one hand, I've noticed there's a "Using lime for regression" notebook (for LimeTabularExplainer), but on the other, the LimeImageExplainer seems to work with classification only (at least judging by the API, I haven't gone through the details yet).

How big a problem do you think this is? What would need to be done?

[marcotcr]

Hey, sorry for the delay in responding. Great question. We do support regression in the TabularExplainer, but I am not convinced it is interpretable enough to be useful at this point. I haven't really played a lot with regression, so that could be it.

For images, I think the problem I listed (features with different scales) would not really be an issue, since we assume that the features are binary. It should be easy to make LimeImageExplainer work with regression. In fact, I think it should just work if you pass a regressor instead of a classifier. Whether or not the explanation will be useful will depend on a few factors:

1. Is the perturbation distribution meaningful enough in this case? I.e. can we capture the model's behavior by hiding parts of the image, or is it relying on features that will not be captured by this simple procedure?
2. Related to the previous point: will the weights be meaningful? I.e. is it helpful to know that 'if I hide this part of this frame, the angle will more or less change by 30 degrees'?

You tell me : )

[MTDzi]

Thanks for your answer!

Indeed, by passing a regressor instead of a classifier I do get an explanation:

where the original image looks like this:

That's actually not bad -- it makes sense that the borders of the road are relevant. But what I'm struggling with is that a large portion of the road itself seems to be somehow relevant. I'm guessing that's because the superpixels that were highlighted were "glued" to the boarders. It might be that, but also it might be the fact that the segments look like this: meaning: they're huge.

But regarding your concerns:

1. That's a valid point. My assumption so far was that the borders of the road pay a role (and nothing else should), so without a border of the road the model is bound to fail in predicting the right angle. Let's stick to this assumption for now.

2. I think that's valuable. To elaborate: what I would like to see is a kdeplot (like those here) overlaid on the original image, showing how much parts of that image "contribute" to the angle predicted by the model.

Now, I took a look at LimeImageExplainer.explain_instance and I thought: "If I generated segments a little differently, I might get many tiny segments, each with its corresponding weight, so that I could combine them into a very rough-looking kdeplot". Also: "What if I used a spectral clustering algorithm to generate those segments (like here)?".

So my question is: do you think injecting a different image segmentation technique into LIME (for my own purposes, I'm not sure anybody else would use this feature) might be the way to go? For the purpose of finding relevant borders, instead of superpixels that contain those borders?

BTW, I was inspired by the functionality presented here where the relevant parts of the image are boarders, not patches (superpixels) of the image. I want to get something similar (but obviously better ;)) in LIME.

[marcotcr]

Yes, these segments are too big to get a useful explanation. You can generate segments in all sorts of ways - the more finer grained they are, the more samples you will need to get a useful explanation. I would suggest trying out different segmentations until you get something that makes sense for your purposes. We already allow you to define your own segmentation function

Also notice that when we 'hide' a superpixel, we replace it with the mean of the image by default. It seems that in your case it would be better to replace it by some neutral color (maybe black?), using this parameter.

[MTDzi]

Thanks, that's very helpful.

I think that's all I need to explain the behavior of the car, I'll let you know if I get something valuable.

[NexusHero]

What do you mean by passing a regressor?

I just predict facial keypoints for images and i try this:

my model has the input shape (96,96,1).

explanation = explainer.explain_instance(faces[0], model.predict, top_labels=5, hide_color=0, num_samples=1000)

Would this work with "model.predict" or i have to pass somthing else?

[ishita-gupta0810]

I had the same doubt and what I found was that it does work directly by passing the regressor, just make sure use explanation.top_labels[0] as there is only one label. Though as mentioned in this thread, the explanation might not be as meaningful as in the case of classification

[ikaroso]

thank u guys. the discussion here is so valuable for me

[scchess]

Does it mean a softmax classification of 2 labels is better than a regression here?