Dataset Distillation aims to distill an entire dataset's knowledge into a fewsynthetic images. The idea is to synthesize a small number of synthetic datapoints that, when given to a learning algorithm as training data, result in amodel approximating one trained on the original data. Despite recent progressin the field, existing dataset distillation methods fail to generalize to newarchitectures and scale to high-resolution datasets. To overcome the aboveissues, we propose to use the learned prior from pre-trained deep generativemodels to synthesize the distilled data. To achieve this, we present a newoptimization algorithm that distills a large number of images into a fewintermediate feature vectors in the generative model's latent space. Our methodaugments existing techniques, significantly improving cross-architecturegeneralization in all settings.
URL
Affiliations
Abstract
Translation (by gpt-3.5-turbo)
Summary (by gpt-3.5-turbo)