search

Zhendong-Wang / Patch-Diffusion

Apache License 2.0
63 stars 11 forks source link

Memory issue when training with imagenet #5

Open sundeco opened 6 months ago

sundeco commented 6 months ago

I am trying to train a network with the full imagenet dataset, but I ran out of all 100+ GB of my memory after it trained for about 50kimg. I am wondering if any tricks were used with the dataset or code in order to avoid this problem when training with imagenet in the paper? Currently I just have all the 1.2 million images in one massive zip file.

Zhendong-Wang commented 6 months ago

Sorry for the late reply. You may need to set this --cache=False, which is by default True. @click.option('--cache', help='Cache dataset in CPU memory', metavar='BOOL', type=bool, default=True, show_default=True)