search

mudler / LocalAI

:robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.
https://localai.io
MIT License
21.65k stars 1.66k forks source link

feature: CUDA support #279

Closed mudler closed 5 months ago

mudler commented 1 year ago
localai-bot commented 11 months ago

:warning::warning::warning::warning::warning:

Hi! I'm a bot running with LocalAI ( a crazy experiment of @mudler ) - please beware that I might hallucinate sometimes!

_but.... I can also be funny or helpful :smilecat: and I can provide generally speaking good tips or places where to look after in the documentation or in the code based on what you wrote in the issue.

Don't engage in conversation with me, I don't support (yet) replying!

:warning::warning::warning::warning::warning:

The issue seems to be related to the RWKV package not being installed correctly or not being used in the correct way. To fix this, you can try the following steps:

  1. Make sure that the RWKV package is installed correctly by running the following command: 
    $(GOCMD) mod download
  2. Check if the RWKV package is being used in the correct way in the code by looking at the Load method of the LLM struct. It should be using the rwkv.LoadFiles function to load the model file and create a new RWKV state object.
  3. If the issue still persists, try reinstalling the RWKV package by running: 
    $(GOCMD) mod uninstall rwkv
$(GOCMD) mod install rwkv
  4. If the issue is still not resolved, please provide more information about the specific error message or behavior that is being observed.

Sources:

mudler commented 5 months ago

we can close this now, as GPU is now supported