Support for custom vectorization in LimeTextExplainer

marcotcr / lime

Lime: Explaining the predictions of any machine learning classifier

BSD 2-Clause "Simplified" License

11.64k stars 1.81k forks source link

Support for custom vectorization in LimeTextExplainer #678

Open TheLastProgramer opened 2 years ago

TheLastProgramer commented 2 years ago

I have noticed that inside Lime Text Explainer, correct me if I am wrong, original text and its perturbed data is converted to binary vector form and there after the distance metric is used to calculate the distance between them. Can we use word embeddings to vectorize the perturbed data and then calculate the cosine distance between them? Will it make any difference if we calculate distances using embeddings rather than default that is currently used?

TheLastProgramer commented 2 years ago

Is there any information/update that can help?

TheLastProgramer commented 2 years ago

Any updates?