-
Hi,
Your code used llma2 chat offline LLM. But, I wanted to use alternative offline LLMs such as huggingface's distilbert or roberta or albert. Do you have any suggestion for those LLMs to apply on …
-
We are trying to launch codegeex4-all-9b Using vllm following the CodeGeeX4 github:
https://github.com/THUDM/CodeGeeX4?tab=readme-ov-file#vllm
The scripts are as following:
codegeex_offline_examp…
-
### 📚 The doc issue
I did see the official documentation contains the offline inference chat function. But I still get the Attribute Error where LLM object does not have chat() attribute. Has this be…
-
### Your current environment
The output of `python collect_env.py`
```text
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A…
-
I would like to have autotagging with LLM support, firstly we should have Ollama support for offline tagging and for privacy reasons and we could add other services too, I think I can work on this.
…
-
Mantella sometimes tries to pull LLM encodings from online even when it is running offline. A workaround needs to be made for this to not search online if running offline, or a better solution would b…
-
Ubuntu22.04,kernel 5.15.0 with 4 * Arc A770 on Xeon(R) w9-3495X
$ clinfo
Driver Version 24.22.29735.27
script
```
source /opt/intel/oneapi/2024.0/oneapi-vars.sh…
-
Platform: Core ultra 7 165H iGPU
Model: Qwen/Qwen2-7B-Instruct
Following the steps on https://testbigdldocshane.readthedocs.io/en/perf-docs/doc/LLM/Quickstart/vLLM_quickstart.html#
when runni…
-
### Describe the bug
When using a custom profile with `claude-3-5-sonnet-20240620`, the code generated is never complete and always results in an error. This usually occurs after a `:` or a number bu…
-
Thanks for the amazing work, of course for a brand new beginner, it takes some times to setup the entire environment.
As readme already said the current best model to use is the gpt4o model. which ma…