issues
search
Leeroo-AI
/
mergoo
A library for easily merging multiple LLM experts, and efficiently train the merged LLM.
https://www.leeroo.com/
GNU Lesser General Public License v3.0
358
stars
19
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Add shortcut for MLP hidden state computation.
#18
jacklanda
opened
2 weeks ago
0
[Feature] Support New Arguments for Expert Routing Policies.
#17
jacklanda
opened
1 month ago
9
Possibility to integrate multiple model types?
#16
sammcj
closed
1 month ago
2
[Fix] Fix the Error of q, k, and v states must have the same dtype when using flash attention forward.
#15
jacklanda
closed
1 month ago
0
phi2
#14
nlee-208
closed
1 month ago
0
LoRA MoE with k_proj, up_proj, down_proj
#13
aksh555
opened
1 month ago
5
It will be a good idea to support T5 series model.
#12
svjack
opened
1 month ago
1
Improvement: Use bfloat16
#11
PhilipMay
closed
1 month ago
1
Phi3 merge issue
#10
PhilipMay
closed
1 month ago
10
Phi3 support
#9
alirezamshi
closed
2 months ago
0
ModuleNotFoundError: No module named 'mergoo.composers'
#8
if001
closed
2 months ago
1
merge failing
#7
saucam
closed
2 months ago
8
How many H100-80GB devices Needed for Merging and Tuning a Llama 3-based MoE Model?
#6
jacklanda
closed
2 months ago
2
Llama3 support
#5
alirezamshi
closed
2 months ago
1
How to set 'router_layers' when making BERT MoE?
#4
gauss5930
closed
2 months ago
4
Update readme.md
#3
eltociear
closed
2 months ago
1
Developer Documentation to contribute new models
#2
abhinav-kashyap-asus
closed
2 months ago
4
Adding Mixture-of-Adapters features
#1
alirezamshi
closed
2 months ago
0