search

libffcv / ffcv

FFCV: Fast Forward Computer Vision (and other ML workloads!)
https://ffcv.io
Apache License 2.0
2.8k stars 180 forks source link

Can not run in parallel #256

Closed Kranito closed 1 year ago

Kranito commented 1 year ago

When I want to start some model training programs at the same time, the program will shut down and the error reports Memory Error. What should I do if I want to run a few ffcv programs in parallel?

Kranito commented 1 year ago

When I want to use multi-threading to run different models on different subsets of one big dataset, after creating these threads, this project will report ''Numba: Attempted to fork from a non-main thread, the TBB library may be in an invalid state in the child process.'' So what should I do if I want to run some different threads in one process?