Support for Qwen-v1.5-110b

pcg-mlp / KsanaLLM

Other

286 stars 30 forks source link

Support for Qwen-v1.5-110b #4

Closed Superskyyy closed 5 months ago

Superskyyy commented 5 months ago

Hi, does KsanaLLM currently support Qwen-v1.5-110b? If not is it possible to adapt the framework to do so? Thanks!

pcg-mlp commented 5 months ago

Qwen-v1.5-110b is not verified on KsanaLLM right now. We will check this and reply later.

Superskyyy commented 5 months ago

Qwen-v1.5-110b is not verified on KsanaLLM right now. We will check this and reply later.

We are very interested in adopting KsanaLLM for larger models. Thanks for the update!

xingjinglu commented 5 months ago

Qwen-v1.5-110b is not verified on KsanaLLM right now. We will check this and reply later.

We are very interested in adopting KsanaLLM for larger models. Thanks for the update!

The verification is finished. KsanaLLM can run Qwen-v1.5-110b on 8*A100 GPUs.

Superskyyy commented 5 months ago

Qwen-v1.5-110b is not verified on KsanaLLM right now. We will check this and reply later.

We are very interested in adopting KsanaLLM for larger models. Thanks for the update!

The verification is finished. KsanaLLM can run Qwen-v1.5-110b on 8*A100 GPUs.

I assume it also runs on 8 Ascend NPUs?

whitelok commented 5 months ago

Qwen-v1.5-110b is not verified on KsanaLLM right now. We will check this and reply later.

We are very interested in adopting KsanaLLM for larger models. Thanks for the update!

The verification is finished. KsanaLLM can run Qwen-v1.5-110b on 8*A100 GPUs.

I assume it also runs on 8 Ascend NPUs?

not support. only support one NPU now. multiple npu scheduled in June/July.

Superskyyy commented 5 months ago

Qwen-v1.5-110b is not verified on KsanaLLM right now. We will check this and reply later.

We are very interested in adopting KsanaLLM for larger models. Thanks for the update!

The verification is finished. KsanaLLM can run Qwen-v1.5-110b on 8*A100 GPUs.

I assume it also runs on 8 Ascend NPUs?

not support. only support one NPU now. multiple npu scheduled in June/July.

Thanks!