-
### Jan version
1
### Describe the Bug
My System (pop os) is having 32GB RAM, 12 GB RTX 4080. for any model even for a singe layer offloading on GPU, i am getting below error although BLAS = 1 and…
-
Whether GroundingDINO can support TensorRT-LLM multimodal ？
[TensorRT-LLM multimodal ](https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/multimodal/README.md)
-
Hello,
Is there a way to create an embedding model object (with `LiteLLMEmbeddingModel()` i guess) from a locally served embedding model.
For more precision I run in parallel :
- 'Mixtral-7x…
-
### Your current environment
hardwark： A800
Driver Version: 535.54.03 CUDA Version: 12.2
vllm commit d3a245138acb358c7e1e5c5dcf4dcb3c2b48c8ff
model qwen72B
### Model Input Dumps
_No response…
-
@kennethreitz started engaging [CodeRabbit](https://github.com/coderabbitai) on this repository the other day.
About leveraging LLM bots for code reviews, this one just appeared on my radar through…
amotl updated
4 weeks ago
-
I have attempted numerous times and several with the help of ChatGPT4o to correct the error of File "K:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_LLM_party\llm.py", line 16, in
impor…
-
### Describe the bug
I'm attempting to use some of my local LLMs on Ollama in this fork and everything works great. There aren't any issue with the execution itself. However I'm running into an iss…
-
is there any support for open-sourced llms like llama3.1 without using the closed source ones like gpt4o & claude?
-
Users should be given the option to choose an LLM provider (e.g, OpenAI, Ollama) and to choose a specific model from the provider. For now, OpenAI is the only provider and gpt-4o-mini is the only mode…
-
Thank you for putting this together. Any resources on running vision language models in Swift ?