microsoft DeepSpeed-MII issues

microsoft / DeepSpeed-MII

MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.

Apache License 2.0

1.91k stars 175 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

How to use data parallelism in multi gpus inference

#546 hhf-hu opened 7 hours ago
0
Issue: Multi-node and Multi-GPU Inference Problems with DeepSpeed MII

#545 lcnmzz00 opened 2 days ago
0
Please clarify structured output support

#544 MRYingLEE opened 3 days ago
0
Bug: Removal of mii.pydantic_v1 broke entrypoint scripts

#543 KMouratidis opened 1 week ago
3
Update transformers

#542 loadams opened 1 week ago
0
Updating transformers issue with bloom models

#541 loadams opened 2 weeks ago
0
Updating transformers issue with zero-shot-image-classification

#540 loadams opened 2 weeks ago
0
Update version.txt

#539 loadams closed 3 weeks ago
0
Update clang-format version to match DeepSpeed

#538 loadams closed 3 weeks ago
0
Update path triggers that were incorrect before

#537 loadams closed 3 weeks ago
0
Non-persistent example fails with KeyError

#536 jjaymick001 closed 3 weeks ago
1
Update CODEOWNERS

#535 loadams closed 3 weeks ago
0
Update labels to acquire new runners

#534 loadams closed 3 weeks ago
0
Update docker container version

#533 loadams closed 3 weeks ago
0
Logits Processors

#532 psitronic opened 1 month ago
0
need help understanding profiler in deespeed mio

#531 krishnanpooja opened 1 month ago
0
Deepspeed mii library issues

#530 gayatripadmani closed 3 weeks ago
2
DeepSpeed with Phi-3-mini-128K-instruct does not generate `<|endoftext|>` token

#529 shubhanshu786 opened 1 month ago
1
Repeated token generation with Phi-3-mini for longer context

#528 shubhanshu786 opened 1 month ago
0
LoRA Support

#527 bagelbig opened 1 month ago
0
deepspeed MoE all_to_all communication

#526 miaomiaoma0703 opened 2 months ago
0
multi model deployment

#525 whcjb opened 2 months ago
1
Fix missing pydantic updates in legacy mii code

#524 loadams closed 2 months ago
0
Question About Offloading and Recomputation

#523 lxnlxnlxnlxnlxn opened 2 months ago
0
Configuration setting to pass parameters to tokenizer while encoding and decoding

#522 krishnanpooja opened 2 months ago
0
OpenAI server fails

#521 nivibilla opened 3 months ago
1
Update version.txt after 0.3.0 release

#520 loadams closed 3 months ago
0
Update supported model list

#519 tohtana closed 3 months ago
0
By default does deepspeed mii use bf16 dtype or fp16?

#518 krishnanpooja opened 3 months ago
0
Confirm PyDantic v2 update passes DS tests

#517 loadams closed 3 months ago
0
FileExistsError: [Errno 17] File exists: '/tmp/mii_cache' ` on generate function call

#516 krishnanpooja opened 3 months ago
0
Fix scheduling for non-persistent pipeline

#515 tohtana closed 3 months ago
0
Can't use Llama 3.1 with MII, ImportError: cannot import name 'Conversation' from 'transformers'

#514 chuyuanli closed 3 months ago
1
non-persistent example doesn't work on Mixtral-8*7B-v0.1

#513 tang-t21 opened 3 months ago
0
Support latest changes in transformers

#512 loadams opened 3 months ago
0
Update version.txt

#511 loadams closed 3 months ago
0
Pin to use a specific version of transformers

#510 loadams closed 4 months ago
0
Test adding torchvision to fix CI failures

#509 loadams closed 4 months ago
0
Update workflow task to use Ubuntu 22.04

#508 loadams closed 4 months ago
0
Update MII to switch from modelid to id

#507 loadams closed 3 months ago
0
non-persistent simple example does not work

#506 mohbay opened 4 months ago
5
Dummy data loading?

#505 guqiqi opened 4 months ago
0
Client cannot find deployment error

#504 heiseon opened 4 months ago
0
CUDA device rank in mii.pipeline

#503 RealPolitiX opened 4 months ago
0
Import Error, not compatible with transformer package

#502 tang-t21 closed 3 months ago
4
deepseed-mii支持多节点推理么

#501 JKYtydt closed 2 weeks ago
2
Run pydantic 2 tests with updated DeepSpeed branch

#500 loadams closed 4 months ago
0
[QUERY] Expert Parallelism Supported?

#498 Shamauk opened 4 months ago
0
Attempting to flush sequence N which does not exist

#497 aagontuk opened 5 months ago
0
Compute perplexity

#496 Sh1gechan opened 5 months ago
0