Closed defatoraj closed 8 months ago
Hi @defatoraj, by default, using LLMLingua requires NVIDIA CUDA to be enabled. You can switch to CPU mode with the following settings.
from llmlingua import PromptCompressor
llm_lingua = PromptCompressor(device_map="cpu")
Hi, i apologize for the late response. It is still not working for me. Executed: from llmlingua import PromptCompressor llm_lingua = PromptCompressor(device_map="cpu") And got this error: Canceled future for execute_request message before replies were done The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click here for more info. View Jupyter log for further details.
Hello!,
Getting the "Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx" error when trying to execute the below code.
from llmlingua import PromptCompressor llm_lingua = PromptCompressor()