gumityolcu
commented
5 hours ago Hello,
Thank you for your message! First of all, your understanding of the implementation is correct. Just some details about the contents of csv file to be completely clear: we have a line for each sample, we first input final hidden layer feature values, and the final value should be the label.
You are right that the method is agnostic to data modality, as it only relies on computed features, only restriction is that the network should end with a fully connected classifier layer.
It seems like the explanations at least pass some sanity checks, like choosing samples from the same class as the prediction class as proponents, however explanations in terms of text samples are hard to digest. DualView explanations correlate with final layer feature similarity, and therefore directly reflect the penultimate representations in its explanations.
In principle, you can do any Training Data Attribution method to your problem. You could probably also implement Representer Points which is also easily usable in terms of computational requirements. However, since they also rely on final layer feature similarity, I would expect suboptimal outputs from it too.
Specifically for NLP, people used NLP tasks in TRAK and Kronfluence for example. Discussions in the related parts of these papers might give some ideas. However these gradient based methods require high computational resources.
We are currently working on applying DualView to the text domain too, but we don't have results yet, these will be put in and discussed in the next iteration of the paper.
hhf-hd
Hi Yolcu,
First of all, thank you for your excellent work on "DualView: Data Attribution from the Dual Perspective." The approach and results you demonstrated on MNIST and CIFAR datasets are both impressive and compelling.
I have been studying the paper and trying to understand its core methodology. From my understanding, the method primarily involves training a surrogate multi-class SVM to obtain the Lagrange multipliers and parameter W, which together form the attribution scores linking training data to test samples. The input to the method appears to be a data.csv file containing feature vectors and corresponding labels. Could you confirm if this understanding is correct, or clarify any points I may have misunderstood?
Additionally, I have tried to extend your code to the NLP domain by applying it to the AG-News dataset for text classification. For reference, the dataset includes four categories:
Class 0: World Class 1: Sports Class 2: Business Class 3: Sci/Tech I have attached examples in the following files for your reference:
errorSample.txt: Contains samples where predictions were incorrect. sample.txt: Contains correctly classified examples. However, the results I obtained do not seem as compelling or convincing as those reported for image datasets in the paper. This raises a few key questions:
Could this discrepancy stem from potential errors in my implementation, or might the method inherently be less suitable for text classification tasks? Since the input consists of feature vectors and labels in the data.csv format, I assumed the method would be agnostic to the domain.
Could you please share your thoughts on its applicability to NLP tasks?
Compared to image classification, text classification might lack the same level of intuitive and objective interpretability. Could you recommend any effective methods for improving interpretability in text classification? I am particularly interested in approaches that clearly and visually demonstrate relationships between
examples, especially in the context of the given samples.
This issue has been challenging, and I would greatly appreciate any insights you could provide. Thank you for your time, and I look forward to your response!
Best regards
errorSample.txt
sample.txt