search

Nota-NetsPresso / shortened-llm

Compressed LLMs for Efficient Text Generation [ICLR'24 Workshop]
63 stars 8 forks source link

Using shortened-llm for Instruct models #18

Open sriyachakravarthy opened 1 week ago

sriyachakravarthy commented 1 week ago

Hi! Can we use shortened-llm to prune Instruct models?

lifelongeeek commented 1 week ago

@sriyachakravarthy Of course. Please see discussion with similar issue in https://github.com/Nota-NetsPresso/shortened-llm/issues/16.

sriyachakravarthy commented 1 week ago

Thank you, @lifelongeeek ! Quick questions, what does --no_instruction argument do in the lora_retrain? and what should we exactly pass for --resume_from_checkpoint argument to continue fintetuning ?