search

dvlab-research / LongLoRA

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)
http://arxiv.org/abs/2309.12307
Apache License 2.0
2.62k stars 274 forks source link

Can LongLoRA be used for incremental pre-training? #152

Open Zheng-Jay opened 11 months ago

Zheng-Jay commented 11 months ago

Can LongLoRA be used for incremental pre-training? If so, how can I do that?