search

GeorgeCazenavette / mtt-distillation

Official code for our CVPR '22 paper "Dataset Distillation by Matching Training Trajectories"
https://georgecazenavette.github.io/mtt-distillation/
Other
395 stars 55 forks source link

how to use images？ #11

Closed Fduxiaozhige closed 2 years ago

Fduxiaozhige commented 2 years ago

hello, i wanna know how to use distilled images. I used distilled images to train a new network, but the accuracy was terrible(10% on cifar10). So, can these images be used to train a new network? if not, what's the meanning of these images. if these images can train a new network, can you share me the network architecture.

GeorgeCazenavette commented 2 years ago

Can you give a bit more information on what you're doing?

What network are you trying to train?

What synthetic data are you using?

How is your test data being normalized?

Fduxiaozhige commented 2 years ago

Thanks for answer. I used the images from the home page(https://georgecazenavette.github.io/mtt-distillation/tensors/index.html#tensors), and i wanted to use these images(Cifar10_zca_50) to train a new neural network(LeNet). I just wanted to know whether only use these 500images can train a new network well(at least have over 30% accuracy). I just use the distilled images to train a network, and use the network directly to pridict the testset of Cifar10. I didin't normalize any test data. If i have to, can you tell me how to normalize?

Fduxiaozhige commented 2 years ago

If possible, I want to get a code about how to use your distilled images to train a new network, and have a test on the raw testset.

GeorgeCazenavette commented 2 years ago

If you're using the zca images, you'll need to normalize the test set with the zca transform as calculated on the train set.

Otherwise, you'll need to use the standard subtract and divide normalization as calculated on the train set.

These can both be found in utils.py.

I'm at CVPR right now, so I can't really write any custom code for you at the moment. Hopefully you can figure out the normalization by looking in the utils.py file.

Let me know if you have any more questions :)

Fduxiaozhige commented 2 years ago

OK, thanks for answering my questions carefully. I will finish the experiment by myself. I really appreciate all your help.

GeorgeCazenavette commented 2 years ago

Happy to help :)

mukurgupta commented 2 years ago

Hi @GeorgeCazenavette, thanks for sharing this repo. Great work :)

I'm facing similar issues while training the network with CIFAR-10 distilled images and testing on the standard CIFAR-10 test set.

I've tried normalizing the distilled images as you suggested above but seems like the model is not able to learn from the distilled set. I'm using VGG-16 architecture for my experiments. It'd be great if you can please help in figuring out what I might be missing here?

@Fduxiaozhige were you able to figure out how to use the provided images?

Thanks a lot.