search

deep-learning-with-pytorch / dlwpt-code

Code for the book Deep Learning with PyTorch by Eli Stevens, Luca Antiga, and Thomas Viehmann.
https://www.manning.com/books/deep-learning-with-pytorch
4.74k stars 2.01k forks source link

Do cache paths need to be generated each time? #37

Closed jeffj9930 closed 3 years ago

jeffj9930 commented 3 years ago

Hi, This isn't an issue really but a question. Can I create the cache for each chapter only once? That is, if I have a large enough SSD can I run 'prepcache' on ch12, ch13, ch14 and then be able to experiment on each chapter? Currently I have only enough space for a single chapter's cache so I must clear_cache each time I go to a different chapter and redo prepcache. If I use a 2TB SSD I can fit all three chapters. Is this possible or does some database get matched up with the current cache? I see separate folders under cache so I hope so.

Jeff

elistevens commented 3 years ago

Yes, prepping the cache once per chapter and then leaving them as-is should work fine.

The cache is set up per-chapter to minimize the total disk space needed at one time, but as you note, it can be annoying to move back and forth between chapters if space is tight.

Please reopen the issue if you run into any problems!