Beam: v2 Tracker - main

[enricoros]

Extend and improve the Beam feature of Big-AGI. Possible vectors for improvement are below:

• UX:

  • [ ] Respond to single chats (chat bubble in the bottom-left) #553
  • [ ] Looping Beam (Fusion back to Beams) (reqby Luna)
  • [ ] "respond to all" mode in the Beam section - (@Commando on discord) #553
  • [ ] #552
  • [ ] Costs prediction/indication - e.g. https://artificialanalysis.ai/models#pricing
  • [ ] In-chat beam: shows a progress indicator while beaming and merging behind the scenes, then you'll see the assistant output after the merge. there are several options to do it:
  • explain Modal vs Inline beams, in a way that's logical (if doable)
  • could be a persona checkbox (Beam by default, or "Use Beam on all answers: ON")
  • need to figure out a clutter-free version of the UX
  • [ ] #489

• core:

  • [x] Remember the last configuration of models
  • [x] Auto-select the best batch of models by family, the first time
  • [ ] Fix "beam from here's model reuse": #551
  • [ ] Overridable per-Beam model options: #519
  • [ ] Enable beaming with a persona change, in addition to the model. (@cryptonaut4219 mentions on Discord to "beam between the same model with different system prompts")
  • [ ] Enable variations of the beam input - to wiggle around the starting point and seeing the differences.
  • [ ] For instance with a pre-process step of: “Adopt the role of 1 or more subject matter EXPERTs most qualified to provide authoritative, nuanced answer to the following prompt. Improve the prompt provided below based on this expertise and proceed to respond.”
  • [ ] Option to ignore history when doing Beams and Merge
  • [ ] Benchmark Beams vs SOTA on MMLU

• Merges:

  • [x] Auto-Merge option
  • [ ] https://youtu.be/QV6kaNFyoyQ?si=f8-3I-5YerRlx0gH
  • [ ] "Differences" merge - shows what's unique about each response (could be separate from Compare, or similar)
  • [ ] "Sliders" merge - weight mix with: accuracy vs creativity, relevance, completeness, logical flow, clarity, coherence, originality, currency, and depth of insight, or something like it
  • [ ] "Compare" merge, add model names: #513
  • [ ] Option: Design prompts to provide clear justifications for their choices
  • [ ] Option: Forecast the potential value of responses in addressing the user's goal for a query

Add the following suggestion: maybe have a verbosity drop down to pick from 5 fusion options, something like this:

VERBOSITY

V=1: extremely terse V=2: concise V=3: detailed V=4: comprehensive V=5: exhaustive and nuanced detail with maximum comprehensive depth and breadth

[keithclift24]

Further explanation on the "Differences" button idea:

Could have a differences button look strictly for discrepancies in facts, accuracy, inconsistency. Just showing differences, so you can weed out bad model responses. Then on to compare button for comparing the quality with criteria of remaining. / So a good user process flow = Write query-> Beam from here-> Use decides/selects models-> Run models-> Click "Differences button" [new] -> Compare button -> Remove model responses if way below others ratings -> "Guide" button [optional]-> Then Fusion or Custom

[keithclift24]

Further explanation on the Verbosity idea:

Within existing slash commands add "/v1", "/v2" etc. or as a hamburger menu option selection, append to the beginning of a user query... /v1: "Response must be extremely terse" /v2: "Response must be concise" /v3: "Response must be detailed" /v4: "Response must be comprehensive" /v5: "Response must be extremely exhaustive with nuanced details, yielding maximum depth and breadth"

[enricoros]

Thanks for the precious definitions @keithclift24

[enricoros]

@keithclift24 related to the verbosity, I want to address this with what I called the "mix-in" prompt framework. How would you feel if you could set the level of verbosity of the current persona (and the prepend /vX to alter it at any time)

[keithclift24]

@keithclift24 related to the verbosity, I want to address this with what I called the "mix-in" prompt framework. How would you feel if you could set the level of verbosity of the current persona (and the prepend /vX to alter it at any time)

I think I am following, sounds good!

[dagelf]

Beaming with different temperatures can be even more useful.

[enricoros]

Beaming with different temperatures can be even more useful.

Very true, good point. Need some temperature override, without making the UI too complex.

Maybe I can add an option to show temperature overrides toggles.