search

NUS-HPC-AI-Lab / DATM

ICLR 2024, Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matching
https://gzyaftermath.github.io/DATM/
89 stars 7 forks source link

VRAM #8

Closed lSkyPassion closed 5 months ago

lSkyPassion commented 5 months ago

Hello, I encountered resource limitations while running experiments. It seems that CIFAR100's IPC10 requires 240GB of VRAM. I noticed that your paper mentioned using four A100 cards; is that really enough? I haven't tried CIFAR100's IPC50 yet, but I estimate it might need more than 320GB. Do you have any solutions? This includes when running tinyImageNet later, which I can't even imagine how demanding it will be. The paper mentioned reducing the synthesis steps, right? Are there any other methods?

GzyAftermath commented 5 months ago
  1. Increasing IPC won't change the VRAM. Only synthetic step and synthetic batch size will.
  2. TESLA