Cannot re-initialize CUDA in forked subprocess

Koed00 / django-q

A multiprocessing distributed task queue for Django

https://django-q.readthedocs.org

MIT License

1.8k stars 274 forks source link

Cannot re-initialize CUDA in forked subprocess #713

Open mkzia opened 1 year ago

mkzia commented 1 year ago

I get the following error when a qcluster worker attempts to use CUDA:

Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

How I can get around this error?

mkzia commented 1 year ago

Celery has the ability to share variables between workers. This might be a work around. But does qcluster have global variables? https://stackoverflow.com/a/70618882