Alpha-VLLM / LLaMA2-Accessory

An Open-source Toolkit for LLM Development
https://llama2-accessory.readthedocs.io/
Other
2.68k stars 170 forks source link

Improved param group support #66

Closed linziyi96 closed 12 months ago

linziyi96 commented 12 months ago

The updated param grouping is primarily to support visual fine-tuning with layer-wise lr decay.

The module is tested internally but not used by the main training script of the public version at this moment.