Closed SuperKam91 closed 3 years ago
Sorry for the delay in responding. You are totally right, we should have computed the distances on the scaled x
and z
. I don't remember if I noticed that we weren't doing that when discretization was on, or if I was just too lazy to fix it. I am definitely too lazy now :)
Great thanks for confirming. Totally understand, but now my mind is at ease :)
First of all I wanted to say I love your algorithm, the accompanying paper, and the blogpost which gives a great high-level intuition of how it works.
I have been studying the implementation of the algorithm for tabular data, particularly for categorical variables, and continuous variables when
discretize_continuous == True
.As far as I can see, when calculating the
distances
to be input to thekernel
which measure how far the perturbed points are from the point of interest, thedistances
are calculated betweenx'
andz'
(using the paper's notation), i.e. the interpretable features, which in this case, are the binary variables representing whether the perturbed point is the same categorical variable/in the same discretized bin of the continuous data, as the point of interest.However, equation two of the paper states that the kernel should be computed on the distance between
x
andz
, i.e. the perturbed point and the original point of interest in the original feature space (for the categorical case, this is just the values of the categorical variables, for the discretized case, this is the (scaled) point sampled from the bins and the truncated normal distribution, and the (scaled) point of interest's value, both in the original feature space).To me, calculating the distance between (scaled)
x
andz
seems much more intuitive, as ultimately, it is the perturbed point's proximity with the point of interest in the original feature space, that gives a measure of the locality of the point in the approximation off
, which thus should be used in the weighting of the fit, rather than in the interpretable space.I appreciate that even in the continuous case, equation two isn't strictly followed, as some scaling occurs apriori, but I think we can all agree this is absolutely necessary. If the discretized case was to calculate the distance between
x
andz
, I would support these being scaled in a similar way.I don't want to go into too much detail, but another way to think about the
discretize_continuous
case is to, instead of thinking of the interpretable variablez'
as a binary variable, think of the interpretable functiong
as being a piecewise-continuous function with a domain of the original feature space. Something along the lines of:This is akin (I think) to the way the discretized variables are treated in the algorithm: learning a regression on binary variable
z'
:g(z') = az' + b
, wherez'
is the interpretable variable pertaining to whether the perturbed pointz
lies in the same discretization bin asx
.Now, with the
g(z)
formulation, since bothf
andg
are defined with domains of the original feature space, then it only makes sense to calculate the distance in this space (betweenx
andz
).I appreciate this modification won't change the results of the algorithm much, it was more just a case of cementing my understanding, and hearing your thoughts on the matter.
Thanks in advance for your input!