FasterDecoding Medusa issues

FasterDecoding / Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

https://sites.google.com/view/medusa-llm

Apache License 2.0

2.19k stars 147 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

is_flash_attn_available has been renamed in transformers.utils

#121 simrathanspal opened 2 hours ago
0
Update medusa_introduction.ipynb

#120 simrathanspal closed 6 hours ago
0
[Retraining] Use Liger Kernel to avoid multi-head logits materialization and scale the context length by N times

#119 ByronHsu opened 1 week ago
1
Training code is not working

#118 ksajan opened 1 week ago
2
Instruct data format

#117 orhan6116 opened 3 weeks ago
0
Are Medusa Heads computed in parallel or serially?

#116 userljz opened 1 month ago
0
jinja2.exceptions.UndefinedError: dict object has no element 0

#115 LLLL114 opened 1 month ago
1
updated medusa models in huggingface?

#114 hustxiayang opened 1 month ago
0
[ISSUE] The Pull Request at https://github.com/FasterDecoding/Medusa/pull/97 from Narsil/medusa2 needs to be rolled back.

#112 super-ahn opened 1 month ago
0
do you support Amd gpu -- rocm ??

#111 amd-maheshs3 closed 2 months ago
0
Errors occurred during the environment and training

#110 blacker521 closed 2 months ago
2
train_legacy.py: try to fix indices bug in preprocess.

#109 k-l-lambda opened 2 months ago
0
Does Medusa support beam search decoding strategy?

#108 xs229 opened 2 months ago
0
The implementation of stage 2 with axolotl

#107 boxiaowave opened 3 months ago
0
PPL compute

#106 yuyangxie96 opened 3 months ago
0
Fix TGI's medusa link

#105 fxmarty opened 3 months ago
0
Containerization with Dockerfile to setup medusa

#104 gangooteli opened 3 months ago
0
Fix for removing LM_HEAD and upgrading Medusa v2

#103 tgaddair closed 3 months ago
0
Conversation roles must alternate user/assistant/user/assistant/

#102 gangooteli opened 3 months ago
0
[bug] fix preprocess function

#101 xiezipeng-ML opened 4 months ago
0
Using Medusa with Whisper

#100 AvivSham opened 4 months ago
5
Token-wise the same generalization?

#99 Ageliss closed 3 months ago
2
ImportError: cannot import name 'is_flash_attn_available' from 'transformers.utils'

#98 imneov opened 4 months ago
1
Creating medusa2.

#97 Narsil closed 4 months ago
1
Is there a bug in gen_model_answer_baseline.py?

#96 qspang opened 4 months ago
1
Medusa Training Loss

#95 TomYang-TZ opened 5 months ago
5
train medusa stage-2

#94 smartliuhw opened 5 months ago
1
mistral.json

#93 Git-L1 opened 5 months ago
0
which dataset should i use when training medusa heads with llama2 7b

#92 tu2022 opened 5 months ago
0
Cant it support chatgllm?

#91 PeterXiaTian opened 5 months ago
0
HYDRA support?

#90 arunpatala opened 5 months ago
0
Misleading Name LLM Name MEDUSA

#89 Pittconnect opened 6 months ago
0
about Medusa mask details

#88 dhcode-cpp closed 6 months ago
0
Why medusa-2 train llama2 with no such great improvement?

#85 MeJerry215 opened 6 months ago
2
release medusa-llm v1.0

#84 zhyncs closed 6 months ago
1
Adding recipe for other models (non llama, non vicuna).

#83 Narsil closed 6 months ago
0
[Dynamic Batching] Concerns about whether features are not supported using Medusa

#82 Ageliss opened 6 months ago
0
Encounter an CUDA error when set Medusa head

#81 1649759610 opened 6 months ago
0
Support batch size > 1

#80 xwang365 opened 6 months ago
0
Why the speed up of Medusa 1 on vicuna changed?

#79 niyunsheng closed 7 months ago
2
deepspeed support

#78 jiangix-paper opened 7 months ago
0
Is there no way to inference without training?

#77 MoOo2mini opened 7 months ago
3
medusa-2 HF repo has no 'medusa_num_heads' in config

#76 HaebinShin closed 7 months ago
1
How to use the finetuned mistal model for inference with Medusa

#75 pradeepdev-1995 opened 7 months ago
7
Question about Heads warmup

#74 eloooooon opened 7 months ago
1
Medusa 1 and 2 speed up

#73 LotuSrc closed 7 months ago
2
update Community Adoption for RTP-LLM

#72 zhyncs closed 7 months ago
2
V1.0 prerelease

#71 ctlllll closed 7 months ago
0
Training Medusa heads

#70 mmilunovic-mdcs opened 7 months ago
6
OSError

#69 qspang opened 7 months ago
3