TorchMoE MoE-Infinity issues

TorchMoE / MoE-Infinity

PyTorch library for cost-effective, fast and easy serving of MoE models.

Apache License 2.0

107 stars 8 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Question about prefetch implementation.

#31 zzhbrr opened 2 weeks ago
1
FATAL Failed to read file: , 314, , errno: , 2, , msg: , No such file or directory

#30 Hudayday opened 3 weeks ago
1
Fix GPTQ CPU memory consumption

#29 drunkcoding closed 3 months ago
1
CPU memory problem when using gptq quantization

#28 JustQJ closed 3 months ago
0
RuntimeError: CUDA error: invalid device ordinal. When I run script.py, I meet the error below.

#27 Tingberer opened 4 months ago
2
Readme Example not working (MemoryError: std::bad_alloc)

#26 akhauriyash closed 5 months ago
1
Question: Support for Continuous Batching and Asynchronous Requests

#25 Msiavashi opened 5 months ago
1
CUDA extension not installed Error while running readme_example.py

#24 Msiavashi closed 5 months ago
4
Can the MoE-Infinity framework be used in conjunction with the vLLM framework?

#23 alphabewitch opened 6 months ago
1
Dev

#22 drunkcoding opened 7 months ago
0
Add explicit resource release

#21 lausannel closed 7 months ago
0
Fix Wrong Output Stream Emplace Back

#20 lausannel closed 7 months ago
0
Fix Wrong Stream Emplace Back

#19 lausannel closed 7 months ago
0
Arctic Support

#18 drunkcoding closed 6 months ago
0
Support Consecutive Tasks in Open MoE LLM Leaderboard

#17 lausannel closed 7 months ago
0
Output of Mixtral-8*7b is strange

#16 JustQJ closed 7 months ago
2
run on the mutiple gpus

#15 YLSnowy closed 1 day ago
3
Support Open MoE LLM Leaderboard

#14 lausannel closed 7 months ago
0
add readme example & fix peer access

#13 drunkcoding closed 7 months ago
0
Add CI and Automated Rlease Process

#12 lausannel closed 7 months ago
0
Install from pip failed

#11 future-xy closed 7 months ago
2
How to Install it?

#10 MSGitt closed 7 months ago
2
Feature/expert parallel

#9 drunkcoding closed 7 months ago
0
Grok-1 Support

#8 drunkcoding closed 7 months ago
0
add forward and call

#7 future-xy closed 8 months ago
2
Release dev to main

#6 drunkcoding closed 7 months ago
1
Support Constrained Server Memory

#5 drunkcoding opened 9 months ago
0
Introduce Local Server for OpenAI-Compatible APIs (Beta)

#4 future-xy opened 9 months ago
2
First Release of README

#3 drunkcoding closed 9 months ago
1
MoE-Infinity API Proposal

#2 drunkcoding closed 8 months ago
1
TODO for first release

#1 drunkcoding opened 10 months ago
0