Open EdwardChCriollo opened 4 years ago
yes, it is the tensorpack warning,
if you get small memory, you can set config.TRAIN.process_num = 5 config.TRAIN.prefetch_size = 20
these two params to control it,
or you can set a small batchsize
in default config, 5 20 160 * 2 images will hold on your memory, may cost your problem, just tuning the params for dataiter
Thanks a lot, I will try changing these parameters. I have seen an intensive use of ram(16gb) and SWAP(14gb) memory on my system, if i wanna train adding with 300vw dataset(200000 images, without consider data augmentation process), definitly I must have a lot of memory on my system, is that true?.
acutally , 16g memory is enough, use small batchsize and process_num,
WRN Starting a process with 'fork' method is not safe and may consume unnecessary extra CPU memory. Use 'forkserver' or 'spawn' method (available after Py3.4) instead if you run into any issues. See https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods on how to set them.
I received this warning on training stage and consumes all of memory RAM, which is the reason for this?, it seems like a tensorpack library issue but im not sure, any idea to solve this? Tensorpack == 0.9.9 Tensorflow == 2.0![asd](https://user-images.githubusercontent.com/19244785/82973977-68caed00-9f9e-11ea-9ac8-578e557a7972.png)