Algorithm calculating `distances` between `z'` and `x'` rather than `z` and `x` for categorical and `discretize_continuous` features?

[SuperKam91]

First of all I wanted to say I love your algorithm, the accompanying paper, and the blogpost which gives a great high-level intuition of how it works.

I have been studying the implementation of the algorithm for tabular data, particularly for categorical variables, and continuous variables when discretize_continuous == True.

As far as I can see, when calculating the distances to be input to the kernel which measure how far the perturbed points are from the point of interest, the distances are calculated between x' and z' (using the paper's notation), i.e. the interpretable features, which in this case, are the binary variables representing whether the perturbed point is the same categorical variable/in the same discretized bin of the continuous data, as the point of interest.

However, equation two of the paper states that the kernel should be computed on the distance between x and z, i.e. the perturbed point and the original point of interest in the original feature space (for the categorical case, this is just the values of the categorical variables, for the discretized case, this is the (scaled) point sampled from the bins and the truncated normal distribution, and the (scaled) point of interest's value, both in the original feature space).

To me, calculating the distance between (scaled) x and z seems much more intuitive, as ultimately, it is the perturbed point's proximity with the point of interest in the original feature space, that gives a measure of the locality of the point in the approximation of f, which thus should be used in the weighting of the fit, rather than in the interpretable space.

I appreciate that even in the continuous case, equation two isn't strictly followed, as some scaling occurs apriori, but I think we can all agree this is absolutely necessary. If the discretized case was to calculate the distance between x and z, I would support these being scaled in a similar way.

I don't want to go into too much detail, but another way to think about the discretize_continuous case is to, instead of thinking of the interpretable variable z' as a binary variable, think of the interpretable function g as being a piecewise-continuous function with a domain of the original feature space. Something along the lines of:

g(z) = a + b if z \in discretized bin which x belongs to
g(z) = b otherwise 

This is akin (I think) to the way the discretized variables are treated in the algorithm: learning a regression on binary variable z': g(z') = az' + b, where z' is the interpretable variable pertaining to whether the perturbed point z lies in the same discretization bin as x.

Now, with the g(z) formulation, since both f and g are defined with domains of the original feature space, then it only makes sense to calculate the distance in this space (between x and z).

I appreciate this modification won't change the results of the algorithm much, it was more just a case of cementing my understanding, and hearing your thoughts on the matter.

Thanks in advance for your input!

[marcotcr]

Sorry for the delay in responding. You are totally right, we should have computed the distances on the scaled x and z. I don't remember if I noticed that we weren't doing that when discretization was on, or if I was just too lazy to fix it. I am definitely too lazy now :)

[SuperKam91]

Great thanks for confirming. Totally understand, but now my mind is at ease :)