search

610265158 / face_landmark

A simple method for face alignment based on wingloss and mutitask learning :)
Apache License 2.0
251 stars 80 forks source link

WRN Starting a process with 'fork' method is not safe and may consume unnecessary extra CPU memory. #42

Open EdwardChCriollo opened 4 years ago

EdwardChCriollo commented 4 years ago

WRN Starting a process with 'fork' method is not safe and may consume unnecessary extra CPU memory. Use 'forkserver' or 'spawn' method (available after Py3.4) instead if you run into any issues. See https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods on how to set them.

I received this warning on training stage and consumes all of memory RAM, which is the reason for this?, it seems like a tensorpack library issue but im not sure, any idea to solve this? Tensorpack == 0.9.9 Tensorflow == 2.0 asd

610265158 commented 4 years ago

yes, it is the tensorpack warning,

if you get small memory, you can set config.TRAIN.process_num = 5 config.TRAIN.prefetch_size = 20

these two params to control it,

or you can set a small batchsize

in default config, 5 20 160 * 2 images will hold on your memory, may cost your problem, just tuning the params for dataiter

EdwardChCriollo commented 4 years ago

Thanks a lot, I will try changing these parameters. I have seen an intensive use of ram(16gb) and SWAP(14gb) memory on my system, if i wanna train adding with 300vw dataset(200000 images, without consider data augmentation process), definitly I must have a lot of memory on my system, is that true?.

610265158 commented 4 years ago

acutally , 16g memory is enough, use small batchsize and process_num,