-
how to change this code to make it work on 2 11G 2080ti?:
```# make sure you have accelerate and bitsandbytes installed
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = Au…
-
Hello! I was curious if anyone has gotten models like MPT and Starcoder to work with GGML and the M1 specifically using Metal/GPU. Thanks.
-
I wish to chat to tabby like in your demo page.
But from a bash command line..
At the moment I just used the docker container.. so I have /opt/tabby/bin/tabby and tabby-cpu
How to get a prompt?
Zibri updated
2 months ago
-
### sqlcoder
目前sqlcoder在text2sql领域表现不错,在很多数据应用开发中被广泛使用,希望官方可以提供推理支持
-
实测sqlcoder2(基于starcoder)模型的速度比vllm快,但是输出内容与原版模型相差甚远,是否完全是因为不支持beam search的问题?
-
目前第一步数据清洗是与starcoder相同,想学习了解后面是如何过滤掉低质量代码、语法错误或可读性差的代码的。
谢谢!
-
@b4rtaz Hey, thank you for your wonderful work. Could you please offer some details about how to add supported model? For example, how to to convert some ollama models like command+r or starcoder or l…
-
We observed noticeable variability when re-running the FSDP model training script for a small 1.xB llama2 model with fixed seed(s) and same tokens. Below is a snapshot of the evaluation results on thr…
-
I deployed the converted starcoder model to Triton with a world size of 2 and enabled streaming inference with streaming=True. However, I encountered an issue where the rank 1 model is unable to retri…
-
Hi,I am trying to run the fine-tuning code on my computer, but I got KeyError: 'response',the environment is installed according to the README.
Traceback (most recent call last):
File "/home/star…