Getting 'Found no NVIDIA driver on your system ' error.

microsoft / LLMLingua

To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

https://llmlingua.com/

MIT License

4.42k stars 241 forks source link

Getting 'Found no NVIDIA driver on your system ' error. #35

Closed defatoraj closed 8 months ago

defatoraj commented 8 months ago

Hello!,

Getting the "Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx" error when trying to execute the below code.

from llmlingua import PromptCompressor llm_lingua = PromptCompressor()

iofu728 commented 8 months ago

Hi @defatoraj, by default, using LLMLingua requires NVIDIA CUDA to be enabled. You can switch to CPU mode with the following settings.

from llmlingua import PromptCompressor
llm_lingua = PromptCompressor(device_map="cpu")

defatoraj commented 8 months ago

Hi, i apologize for the late response. It is still not working for me. Executed: from llmlingua import PromptCompressor llm_lingua = PromptCompressor(device_map="cpu") And got this error: Canceled future for execute_request message before replies were done The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click here for more info. View Jupyter log for further details.