arcee-ai / mergekit

Tools for merging pretrained large language models.
GNU Lesser General Public License v3.0
4.88k stars 446 forks source link

gracefully pause evolutionary optimization? #364

Open johnwee1 opened 4 months ago

johnwee1 commented 4 months ago

Is there a way to checkpoint progress on evolutionary merging and resume it gracefully at a later date without restarting? i'm dealing with resource constraints that doesn't allow me extensive compute over a single period of time,

nthangelane commented 3 months ago

I think this is currently not possible looking at how the code looks @johnwee1 . it would be interesting to see the config you are running . maybe this is a feature that can be that after each iteration the code can listen for any haults or a config to check every X iterations