search

karthajee / AWS-IF-interpretability

MIT License
4 stars 2 forks source link

Normalizing feature scores produces erroneous results if unnormalized feature scores sum to negative value #1

Open ClemensA4t1qbit opened 2 years ago

ClemensA4t1qbit commented 2 years ago

Hi, thank you so much for the very interesting paper and the great code to go with it.

I have an observation about using the the weighting score as proposed by your paper, in particular the one defined via 

log2(parent / child) - 1

which leads to some features having negative scores.

As a result, in some instances, the rowsum of a given outliers' feature scores can be negative as well. Using a negative value to normalize of course flips the sign of the row, so now what used to be the most important feature becomes the least important feature.

Not sure what the "correct" way to normalize these scores should be. Maybe normalization in such a way is not appropriate in any case, as we want to know which outliers get a "robust" explanation (with overall large feature scores) and which don't (with overall small feature scores).

karthajee commented 2 years ago

Hello, thank you for responding. I apologise for being a bit tardy — this was my master's research back when I was much younger :) — and I don't frequent the repo as much as I should.

Given a split, parent and child are the sizes of the parent and child nodes, which our sample of interest goes through. The edge case would be parent = child (i.e., no splitting happens per se). In all other cases, since splitting partitions the data, child node will always have lesser number of samples than the parent node it came from. Hence, the lowest value of log2(parent / child) -1 would be 0, wouldn't it? i.e., negative scores are not permitted.

Do you have an example in mind where negative scores are produced?

ClemensA4t1qbit commented 2 years ago

Hi and thanks for the reply.

My understanding was that the -1 gets included as a bias or shift so that a "perfectly balanced split" gets a score of exactly zero: If child = parent/2 we get parent / child = 2, the log2 of which is 1.

Now I'm thinking of a bad split where after partitioning the sample of interest ends up in the larger of the two parts. In that case the ratio will be below 2, the log2 will be below 1 and thus with the extra -1 it'll be something negative.