isdsucph / isds2022

Introduction to Social Data Science 2022 - a summer school course https://isdsucph.github.io/isds2022/
MIT License
21 stars 23 forks source link

Safe n_jobs setting in sk learn #32

Open AlexsandarB opened 2 years ago

AlexsandarB commented 2 years ago

Having a bit of trouble understanding n_jobs. It can run processes faster to put it up. I read that n_jobs=-1 will consume all available recourses, but that is highly impractical and can be unsafe due to computer crashing if doing other tasks at the same time. So my question is if there is like a number before that is bench mark for do this fast but don't take everything I own. For example if ram is the limiting factor and python takes a max of 4gb, then you could with 16gb ram set n_jobs to 3 and then still have 4gb ram for other tasks.