NUS-HPC-AI-Lab / InfoBatch

Lossless Training Speed Up by Unbiased Dynamic Data Pruning
318 stars 18 forks source link

can the dynamic method obtain a final compressed small dataset #9

Open zhangxin-xd opened 1 year ago

zhangxin-xd commented 1 year ago

Hi, I have read your impressive paper and have a question about the dynamic manner: can the dynamic method obtain a final compressed small dataset for cross-architecture evaluation?

henryqin1997 commented 1 year ago

It is a good question, we will do some exploration on it. At this moment, my intuition is that if the pruned set becomes stable at later epochs, it can serve as a coreset selection methods (also may count how many times a sample is pruned and then select). But dynamic pruning method can take the benefit of assessing current learning status of samples, so whether it could obtain a final compressed small dataset and whether it would be good for cross-architecture is still to be investigated.

zhangxin-xd commented 1 year ago

Thank you for the reply. I will keep an eye on the updates.