Leeroo-AI mergoo issues

Leeroo-AI / mergoo

A library for easily merging multiple LLM experts, and efficiently train the merged LLM.

https://www.leeroo.com/

GNU Lesser General Public License v3.0

404 stars 25 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Could I make just in time for defect prediction（one of the text classification tasks） with lora-moe?

#22 chhh745 opened 6 days ago
0
ValueError: `rope_scaling`'s type field must be one of ['su', 'yarn'], got longrope

#21 jzyee opened 2 months ago
1
Bump tqdm from 4.66.2 to 4.66.3

#20 dependabot[bot] closed 2 months ago
0
Could model_type support QWEN models?

#19 AllenShow opened 4 months ago
0
Add shortcut for MLP hidden state computation.

#18 jacklanda opened 5 months ago
0
[Feature] Support New Arguments for Expert Routing Policies.

#17 jacklanda opened 5 months ago
9
Possibility to integrate multiple model types?

#16 sammcj closed 5 months ago
2
[Fix] Fix the Error of q, k, and v states must have the same dtype when using flash attention forward.

#15 jacklanda closed 5 months ago
0
phi2

#14 nlee-208 closed 6 months ago
0
LoRA MoE with k_proj, up_proj, down_proj

#13 aksh555 opened 6 months ago
5
It will be a good idea to support T5 series model.

#12 svjack opened 6 months ago
1
Improvement: Use bfloat16

#11 PhilipMay closed 6 months ago
1
Phi3 merge issue

#10 PhilipMay closed 6 months ago
10
Phi3 support

#9 alirezamshi closed 6 months ago
0
ModuleNotFoundError: No module named 'mergoo.composers'

#8 if001 closed 6 months ago
1
merge failing

#7 saucam closed 6 months ago
8
How many H100-80GB devices Needed for Merging and Tuning a Llama 3-based MoE Model?

#6 jacklanda closed 6 months ago
2
Llama3 support

#5 alirezamshi closed 6 months ago
1
How to set 'router_layers' when making BERT MoE?

#4 gauss5930 closed 6 months ago
4
Update readme.md

#3 eltociear closed 7 months ago
1
Developer Documentation to contribute new models

#2 abhinav-kashyap-asus closed 6 months ago
4
Adding Mixture-of-Adapters features

#1 alirezamshi closed 7 months ago
0