-
### Your current environment
The output of `python collect_env.py`
```
Collecting environment information...
PyTorch version: 2.4.0a0+3bcc3cddb5.nv24.07
Is debug build: False
CUDA used to bu…
-
When I run glm-4-9b-chat-Q5_K_M.gguf on the Cuda 12 machine, the API server can be started successfully. However, when I send a question, the API server will crash.
The command I used to start the …
-
def _build_decoder_cell(self, hparams, encoder_outputs, encoder_state,
source_sequence_length):
"""Build a RNN cell with attention mechanism that can be used by decoder."…
-
This is the trouble I'm facing when I run the chatbot.py file. Any idea on what should I do. When I downloaded the file and I'm running it this is what the error is coming up again and again.
This i…
-
### Your current environment
The output of `python collect_env.py`
```text
Your output of `python collect_env.py` here
```
vllm == 0.5.5
FlashInfer==0.1.6+cu121torch2.4
### 🐛 Descri…
wlwqq updated
1 month ago
-
### Your current environment
vllm版本:0.6.2
4500 ada: 24G*4显存
A100: 80G显存
model: yi1.5-34b-chat-16k
### Model Input Dumps
[err_execute_model_input_20241021-023236.zip](https://github.com/user-atta…
-
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ D:\Python\visual-chatgpt\visual_chatgpt.py:1051 in │
│ …
-
Hi,
This is the issue I'm getting after running. Here's the part of the code in which I'm getting issues. I'm using python 3.5 in anaconda and spyder. One more thing, I know I'm using BahdanauAtten…
-
I am not able to understand what is wrong in the code I wrote since i…
-
作者您好！当我在前端输入框输入文本后，ChatGLM-6B模型推理时存在下面的问题：
INFO: Application startup complete.
INFO: 127.0.0.1:33704 - "OPTIONS / HTTP/1.1" 200 OK
Start generating...
The dtype of attention mask (torch.…