-
-
### System Info / 系統信息
ubuntu22
### Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?
- [X] docker / docker
- [ ] pip install / 通过 pip install 安装
- [ ] installation from source / 从源码安装
…
-
-
用 ppl.pmx Export 导出模型,有大量的警告, Warning: The shape interface of opmx::XX(如 ParallelEmbedding、ColumnParallelLinear、Reshape等) type is missing,用转出来的 onnx 格式的文件启动 ppl_llm_server,提示 unsupported op: domain[op…
-
我看文档里只写支持到qwen1.5,但是issue里不少人有用在qwen2上?
-
I have tested the inference speed and memory usage of Qwen1.5-14b on my machine using the example in ipex-llm. The peek cpu usage to load Qwen1.5-14b in 4-bit is about 24GB. The peek GPU usage is abou…
-
qwen# python convert_checkpoint.py --model_dir /code/tensorrt-llm/Qwen1.5-32B-Chat/ --output_dir ./trt_ckpt/qwen1.5-32b/fp16 --dtype float16 --tp_size 4
[TensorRT-LLM] TensorRT-LLM version: 0.11.0.de…
-
# 模型参数支持专区
大家好,PaddleNLP 团队在这里为大家整理了各个模型参数的详细信息,方便大家使用。
## 模型参数
### Base Models
| Model | 0.5B | 1~2B | 3~4B | 6~8B | 13~14B | 30~32B | 50~60B | 65~72B | 110B | >110B |
|:---------:|:--…
-
### Model description
Here is the model description
> gte-Qwen1.5-7B-instruct is the latest addition to the gte embedding family. This model has been engineered starting from the [Qwen1.5-7B](https:…
-
我的测试脚本:
```
NPROC_PER_NODE=8 \
ASCEND_RT_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 \
HCCL_SOME_VARIABLE=value
swift infer \
--model_type '/data2/dxc/Qwen1.5-32B-Chat' \
--load_args_from_ckpt_dir …