-
感谢博主开源~
最近试用了InternLM-XComposer-VL-7b模型,效果很棒,就是推理的速度有点慢,目前使用V100进行推理,显存26G,耗时10s/条,想问下模型有什么推荐的推理加速方法么,还望博主给点建议
另外还尝试了internlm/internlm-xcomposer-7b-4bit模型,相同机器环境,显存从26G降到20G,耗时翻倍20s/条,不知道是不是我哪里设置的…
-
### Checklist
- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
### Describe the bug
使用lmdeploy/turbomind/decode.p…
GZL11 updated
1 month ago
-
"addmm_impl_cpu_\" not implemented for 'Half'
-
**Describe the bug**
What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程,最好有截图)
```
swift infer --model_type internvl2-8b-awq --infer_backend lmdeploy
```
```
WARNING:ro…
-
# Trending repositories for C#
1. [**moom825 / xeno-rat**](https://github.com/moom825/xeno-rat)
__Xeno-RAT is an open-source remote access tool (RAT) developed in C#, providing a …
-
### Checklist
- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
### Describe the bug
我自制了一个基于internlm2-chat-7b的食谱模…
-
I have used cogvlm v7 (through a guy on patreon) to caption 120,000 images and itndies a very good job, it does it even on some uncensored photos (in detail) if I use a special prompt with English a…
-
### Prerequisite
- [X] I have searched [Issues](https://github.com/open-compass/opencompass/issues/) and [Discussions](https://github.com/open-compass/opencompass/discussions) but cannot get the expe…
-
## cli command
```bash
CUDA_VISIBLE_DEVICES=5 xtuner train internlm_chat_7b_qlora_colorist_e5.py
```
## content of `internlm_chat_7b_qlora_colorist_e5.py`
```python
# Copyright (c) OpenMMLab. …
-
···
ValueError: Model architectures ['InternLMXComposer2ForCausalLM'] are not supported for now. Supported architectures: ['AquilaModel', 'AquilaForCausalLM', 'BaiChuanForCausalLM', 'BaichuanForCausa…