-
### Your current environment
The output of `python collect_env.py`
```text
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N…
-
### System Info
ubuntu 20.04
tensorrt 10.0.1
tensorrt-cu12 10.0.1
tensorrt-cu12-bindings 10.0.1
tensorrt-cu12-libs 10.0.1
tensorrt-llm …
-
### Your current environment
```text
The output of `python collect_env.py`
```
### How would you like to use vllm
I'm implementating a custom algorithm that requires a custom generate met…
-
### What happened?
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
/owner/ninth/llama.cpp/ggml/src/ggml-cann.cpp:61: CANN error: E89999: In…
-
here is the summary:
`unsloth/mistral-7b-v0.3-bnb-4bit` with error : ` KeyError: 'layers.0.mlp.down_proj.weight'`
`unsloth/Qwen2.5-7B-Instruct-bnb-4bit` with error: `KeyError: 'layers.0.mlp.down_pro…
-
### System Info
ubuntu 20.04
tensorrt 10.0.1
tensorrt-cu12 10.0.1
tensorrt-cu12-bindings 10.0.1
tensorrt-cu12-libs 10.0.1
tensorrt-llm 0.10.…
-
### Your current environment
```
Collecting environment information...
PyTorch version: 2.2.1+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: …
-
Hello, thanks to your great work!
In `blip2_vicuna_instruct.py`, the `bos_token` of LLM has been changed. Originally, it is '< s >' with idx:1. But after the following code:
```
self.llm_tokenize…
-
Errors as the following:
(.venv) (base) pengxiong@PENGMacPro PDF-Extract-Kit % python pdf_extract.py --pdf demo/demo1.pdf
[2024-07-19 20:17:51,713] [ ERROR] check_version.py:39 - Error fetching …
-
**Problem**
No source are working on kalosm::language: error Unexpected status code: 401
**Steps To Reproduce**
Example code:
```
use kalosm::language::*;
#[tokio::main]
async fn main()…