-
When we apply chat template + tokenize together, there is a single BOS token, as expected. But if we do them separately (as is done in https://github.com/OpenRLHF/OpenRLHF/blob/main/openrlhf/datasets/…
-
Hello Guys,
I'm wondering about performence, which is very strange
on the same server, i ran the same model with query, and the loading time is totally differente between llama-cpp python and ll…
-
### Your current environment
vllm version: 0.4.2
```
CUDA_VISIBLE_DEVICES=6 python -m vllm.entrypoints.openai.api_server \
> --model mistralai/Mistral-7B-Instruct-v0.3 \
> --dtype au…
-
### System Info
OS:
```bash
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="http…
-
### 💻 Operating System
Windows
### 🌐 Browser
Chrome
### 🐛 Bug Description
after cloned repo localhost on buntu I created account tried to validate but after that i got
Error: relation "public.w…
-
### Python -VV
```shell
NOT REQUIRED
```
### Pip Freeze
```shell
NOT REQUIRED
```
### Reproduction Steps
NOT REQUIRED
### Expected Behavior
As per the [documentation](https://docs.mistral.ai/…
-
https://huggingface.co/mistralai/Codestral-22B-v0.1
https://mistral.ai/news/codestral/
-
### 🥰 需求描述
遇到一个需求:用户不主动对ChatGPT发送消息,需要程序去请求ChatGPT,然后将返回的消息显示到消息列表中;
也就是说,需要ChatGPT主动发送消息给用户,而不是用户发送消息给ChatGPT后ChatGPT再回复。
### 🧐 解决方案
所以我去翻了文档,没有找到添加ChatGPT的消息到消息列表的方法,只找到了用来给ChatGPT发送消息的Ref…
-
### Describe the bug
1. The default system msg is irrationally long and no model except gpt4 preview can even load it. It cost a $1 per minute of use sending this 10,000 token msg each time.
2. Afte…
6rz6 updated
3 months ago
-
A recent PR pinned the minimum pyarrow version to `^15.0.0`, but that's a fairly new version -- jan 21, 2024 -- and much of the pydata ecosystem is not there yet, e.g., the latest nvidia rapids is onl…