search

BBC-Esq / VectorDB-Plugin-for-LM-Studio

Plugin that lets you ask questions about your documents including audio and video files.
https://www.youtube.com/@AI_For_Lawyers
284 stars 36 forks source link

Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method #111

Closed mont13 closed 9 months ago

mont13 commented 9 months ago

Hello i have Ubuntu 22.04 and when i tried create vector databses i have this input. NVIDIA-SMI 545.23.08 Driver Version: 545.23.08 CUDA Version: 12.3 NVIDIA GeForce RTX 3090 with 24GB

doh set to "" -- SystemOnly QPainter::end: Painter ended with 4 saved states QPainter::end: Painter ended with 4 saved states QPainter::end: Painter ended with 4 saved states database_interactions.py: Loading documents. document_processor.py: Loading images, if any. concurrent.futures.process._RemoteTraceback: """ Traceback (most recent call last): File "/usr/lib/python3.11/concurrent/futures/process.py", line 256, in _process_worker r = call_item.fn(*call_item.args, *call_item.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/9b7b4009-9d6e-41d3-bad9-7b870a56b0bf/lmstudio/ChromaDB-Plugin-for-LM-Studio/src/document_processor.py", line 46, in choose_image_loader return image_loader.salesforce_process_images() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/9b7b4009-9d6e-41d3-bad9-7b870a56b0bf/lmstudio/ChromaDB-Plugin-for-LM-Studio/src/loader_images.py", line 287, in salesforce_process_images model = BlipForConditionalGeneration.from_pretrained("Salesforce/blip-image-captioning-large").to(device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/miro/Documents/python/chromadb/lib/python3.11/site-packages/transformers/modeling_utils.py", line 2460, in to return super().to(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/miro/Documents/python/chromadb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1160, in to return self._apply(convert) ^^^^^^^^^^^^^^^^^^^^ File "/home/miro/Documents/python/chromadb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 810, in _apply module._apply(fn) File "/home/miro/Documents/python/chromadb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 810, in _apply module._apply(fn) File "/home/miro/Documents/python/chromadb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 810, in _apply module._apply(fn) File "/home/miro/Documents/python/chromadb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 833, in _apply param_applied = fn(param) ^^^^^^^^^ File "/home/miro/Documents/python/chromadb/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1158, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/miro/Documents/python/chromadb/lib/python3.11/site-packages/torch/cuda/init.py", line 284, in _lazy_init raise RuntimeError( RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method """

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/mnt/9b7b4009-9d6e-41d3-bad9-7b870a56b0bf/lmstudio/ChromaDB-Plugin-for-LM-Studio/src/gui_tabs_databases.py", line 16, in run create_vector_db.run() File "/mnt/9b7b4009-9d6e-41d3-bad9-7b870a56b0bf/lmstudio/ChromaDB-Plugin-for-LM-Studio/src/database_interactions.py", line 42, in run documents = load_documents(self.SOURCE_DIRECTORY) # invoke document_processor.py; returns a list of document objects ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/mnt/9b7b4009-9d6e-41d3-bad9-7b870a56b0bf/lmstudio/ChromaDB-Plugin-for-LM-Studio/src/document_processor.py", line 122, in load_documents processed_docs = future.result() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/concurrent/futures/_base.py", line 456, in result return self.get_result() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in get_result raise self._exception RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

BBC-Esq commented 9 months ago

I don't use Linux and have never encountered this error before. The best I can do is give you a link to the troubleshooting I attempted to do. You'll have to read it and modify the script/scripts accordingly. The "entry point" for my program is obviously gui.py since you have to type "python gui.py" to start the program. Here's the link: https://chat.openai.com/share/ddb7bfe8-a1f1-4fbb-8b3c-3de361d9e028

If you'll do this and let me know if it works, I'd appreciate it.

mont13 commented 9 months ago

Hello, as you sent me the link "https://chat.openai.com/share/ddb7bfe8-a1f1-4fbb-8b3c-3de361d9e028", I have updated the file gui.py to fix this problem. You can test it, but it is working well for me. You have to add it to the main script gui.py in the places I mentioned earlier; this is the correct way to fix it.

import multiprocessing

def main():

Set the multiprocessing start method to 'spawn'

multiprocessing.set_start_method('spawn')

# Your existing code here

if name == "main": main()

Miroslav

BBC-Esq commented 9 months ago

Cool, I'll take a look and test it on my system, and if it works, accept the pull request!

BBC-Esq commented 9 months ago

112