-
### What is the issue?
Follow-up of #586
Even though the output is **deterministic** and **reproducible** with a fixed `seed`, a `temperature` set to 0 and a fixed `num_ctx`, the generated outpu…
-
### Your current environment
Collecting environment information...
PyTorch version: N/A
Is debug build: N/A
CUDA used to build PyTorch: N/A
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04 LTS…
-
### Your current environment
driver 1.17
vllm 0.5.3.post1+gaudi117
```text
export VLLM_GRAPH_RESERVED_MEM=0.1
export VLLM_GRAPH_PROMPT_RATIO=0.9
export VLLM_PROMPT_S…
-
When trying to import FastLanguageModel from unsloth on specifically aws sagemaker I get Traceback (most recent call last):
File /opt/conda/lib/python3.10/site-packages/IPython/core/interactivesh…
-
Environment:
Docker 24.0.7 on Ubuntu in WSL 2.1.5.0 on Windows 11
1. Create `Dockerfile.llama`:
```
FROM ubuntu:24.04@sha256:3f85b7caad41a95462cf5b787d8a04604c8262cdcdf9a472b8c52ef83375fe15
…
-
Option to use OpenRouter Models
-
Hello,
I am encountering an issue while testing FlexFlow's LLM module. Below is the code I am using:
`import flexflow.serve as ff
import time
ff.init(
num_gpus=1,
memory_per_gpu=2200…
-
### Your current environment
```tex
The environment is the latest vllm-0.5.4's docker environment, and the command to run is:python3 api_server.py --port 10195 --model /data/models/Mistral-Large-Ins…
-
### Bug Description
When using the aload method of the AsyncWebPageReader and one of the urls is:
- a direct url to a .pdf file
- a url which is redirecting to a .pdf file
then the aload method …
-
硬件信息:
![image](https://github.com/modelscope/swift/assets/48780003/8dd13a05-f881-4eb1-85d0-0f2343c04cfd)
环境信息:
Python 3.11.9
torch 2.1.1
CUDA 12.1
ms-swift 2.2.0.dev0
执行脚本:
```
CUDA_VISIB…