The DARE-TIES experiment.

[David-AU-github]

I just wanted to pass on some "lab" results using dare-ties and mistral nemo.

I created a triple dare-ties merge of 3 pass-through "instruct/fine" models.

Each instruct/fine tune uses the same merge format:

slices:

• sources:
  • model: g:/11b/Mistral-Nemo-Instruct-2407-12B layer_range: [0, 14]
• sources:
  • model: G:/11B/Rocinante-12B-v1.1 layer_range: [8, 24] parameters: scale:
    • filter: o_proj value: 1
    • filter: down_proj value: 1
    • value: 1
• sources:
  • model: g:/11b/Mistral-Nemo-Instruct-2407-12B layer_range: [14, 22] parameters: scale:
    • filter: o_proj value: .5
    • filter: down_proj value: .5
    • value: 1
• sources:
  • model: g:/11b/Mistral-Nemo-Instruct-2407-12B layer_range: [22, 31] parameters: scale:
    • filter: o_proj value: .75
    • filter: down_proj value: .75
    • value: 1
• sources:
  • model: G:/11B/Rocinante-12B-v1.1 layer_range: [24, 40] parameters: scale:
    • filter: o_proj value: 1
    • filter: down_proj value: 1
    • value: 1 merge_method: passthrough dtype: bfloat16

THE DARE-TIES:

models:

• model: E:/MN-Rocinante-12B-v1.1-Instruct
• model: E:/MN-magnum-v2.5-12b-kto-Instruct parameters: weight: .6 density: .8
• model: E:/MN-12B-Celeste-V1.9-Instruct parameters: weight: .38 density: .6 merge_method: dare_ties tokenizer_source: union base_model: E:/MN-Rocinante-12B-v1.1-Instruct dtype: bfloat16

What is interesting here is that EACH TIME I run the "dare-ties" it creates a slightly different or VERY DIFFERENT model, despite no changes in the the models nor the settings.

This shows up in PPL and "real world" tests. PPL range of 7.7327 to 7.8024 ... and that is on just 10 generations.

Real world testing the "core" changes -> wow. Attibute, scale, word choice, sentence structure,... changes across the board.

I am not sure if this is a mistral nemo artifact or not.

From these 10, I did some merging of these using breadcrumbs ; wow. All I can say.

When everything is F32 ... they shine even brighter.

With enough generations + merging of the "best DNA" could create truly legendary model(s).

Just saying - job well done and then some!!!

NOTE: Models for "fine/instruct" and "DARE-TIES" supermerges are posted at my repo.

[CasualDev242]

If DARE-Ties gives dramatically different results each time, maybe I don't understand it correctly, but that sounds less like a good thing and more like a bad thing.

[David-AU-github]

If DARE-Ties gives dramatically different results each time, maybe I don't understand it correctly, but that sounds less like a good thing and more like a bad thing.

This all depends... in my first case it was bad, because I deleted the source and found out the hard way... and it was a great version. That being said, in creating 10+ versions, the "Dna" of each model can be mapped, and these combined creating stronger models with specific attributes while reducing the negative ones.

One of the open questions is: Does this apply to other archs too? Llama2? 3? 3.1? ... And some of the other mergekit methods also involve this same type of "random pruning"... too. I mapped these out after looking at the programming code to verify operations.

A more interesting method or change may be pruning controls for DARE TIES , which limit the range.

[cg123]

Thanks for sharing your results here!

DARE-TIES does have a randomized element, yeah - it's part of the algorithm by design. If you want more reproducible merges you can set a random seed by passing --random-seed <N> on the command line. I usually do when I'm iterating on a recipe that involves DARE.

[David-AU-github]

Thanks for sharing your results here!

DARE-TIES does have a randomized element, yeah - it's part of the algorithm by design. If you want more reproducible merges you can set a random seed by passing --random-seed <N> on the command line. I usually do when I'm iterating on a recipe that involves DARE.

*** Thank you ; that was one of the questions I had ; thanks again ... I think there is so much untapped potential in mergekit yet to be discovered.