-
Running into issues when serving Mixtral 8x7B on 4 x H100 (TP=4) with deepspeed-mii v0.2.3 with all other arguments default in the base image from nvidia `nvidia/cuda:12.3.1-devel-ubuntu22.04`
The …
ghost updated
3 months ago
-
Hi, could you guide me on how to use this with libgdx-based project?
As libgdx project might have several sub-module targeting different platform: `desktop`, `android`, `ios-moe`, etc. Should I put…
-
Hello everyone,
Following an update to Zigbee2MQTT 1.39.00 , I have noticed strange behavior with some of my roller shutters (specifically those with modules recognized as TUYA-TS130F. I have 2 ot…
-
### Your current environment
环境:
torch 2.3.0
vllm 0.5.0.post1
transformers 4.41.2
主要报错情况:
moe小一点的模型 '/data/models/qwen/qwen1.5-2.7Bmoe' 不会出问题
对于大一点的就报错如最下面。
代码:
from vllm.engine.arg_ut…
-
Hardware/Software Info just in case:
- OS: Solus 4.4
- Launcher: Flatpak
- CPU: AMD Ryzen™ 5 3400G
- GPU: Integrated AMD Radeon™ Vega 11 Graphics
- DE: Budgie 10.8.2
- Linux Kernel: 6.5.11-263…
-
Theres google api that is free and idea is that the user use a command along with image url and bot can use reverse image
This person made a bot for that but it can only search anime picture due to…
-
想要的功能可以在这里许愿ww
-
I replaced one of the layers of gpt2 model with a moe layer training with deepspeed_stage_2. However when trying to run convert_to_fp32.py, I run into all sorts of errors. Does the library currently s…
-
Describe the bug
Get an AtrributeError when trying to convert llama3-8B model from HF format to mcore format, the error is below:
`AttributeError: 'Tokenizer' object has no attribute 'vocab_size'`…
-
requirements.txt中是torch 2.0.0;安装的时候和triton 2.1.0 不兼容;
安装时triton改为2.0.0安装;
安装后单独更新安装triton至2.1.0版本;
server可以正常运行,请求时发生错误:
> /root/.triton/llvm/llvm+mlir-17.0.0-x86_64-linux-gnu-centos-7-rel…