search

jgravelle / AutoGroq

AutoGroq is a groundbreaking tool that revolutionizes the way users interact with Autogen™ and other AI assistants. By dynamically generating tailored teams of AI agents based on your project requirements, AutoGroq eliminates the need for manual configuration and allows you to tackle any question, problem, or project with ease and efficiency.
https://autogroq.streamlit.app/
1.31k stars 440 forks source link

A conform map (as configuration) used for model names when exporting from autogroq #41

Closed JaloWalt closed 3 months ago

JaloWalt commented 4 months ago

Reason: I fail to find a model-api that works both in autogroq and in autogen.

The main problem with autogen (seems to be) is that groq-api limits the response-rate to a very low value. Remember you had to do special code also in autogroq?

So, when the export [button-press works nicely] from autogroq to autogen has been done then what do you do with the stuff now in autogen? You can not use the groq-models.

I went as far as using fireworks.ai api-inference (somewhat similar as groq-api) in autogen, but it seems a bit clumsy to need to go through all that was exported from autogroq and then change that (manually) in autogen to some model that (just/maybe/might) work with the autogroq generated stuff.

At this point also an awful heretical blasphemous idea is born: maybe the fireworks.api could be used in autogroq? So that both autogroq and autogen could use the same model.

Well, the meantime: any idea for the least-manual-effort in matching/making the models align in autogroq and autogen for actual immediate-continual-working after the autogroq-export-to-autogen? Maybe defining a default model in/for the export(ed) model to-be in autogen/crewai etc? Or something better: a conform map?

The fireworks.ai allure reason? It has a way higher response-rate limit(s). Actually in the very first contact the account-executive promised to me that the rate could be increased to 2,000 RPM if I so want.

In summary: If you have an/any example of an actual smooth continuing of the the work (in autogen studio) after exported from autogroq then that surely would be appreciated.

As a feature request: A conform map (as configuration) for model names when exporting from autogroq. That would at least resolve the model name differences between various apis.

Regards; Mr. Jalo Ahonen

P.S A list of few (fireworks.ai) models I have tried in autogen studio.

Structure in autogen; for example the first one:

baseurl is = https://api.fireworks.ai/inference/v1 model name is = accounts/fireworks/models/firefunction-v1

(Yeah, that's the [longish] model name(s). Not quite like in autogroq, even when using mixtral.)

$0/M Tokens 32,768 Max Context https://api.fireworks.ai/inference/v1 accounts/fireworks/models/firefunction-v1

$0.9/M Tokens 200,000 Max Context https://api.fireworks.ai/inference/v1 accounts/fireworks/models/yi-34b-200k-capybara

$0.2/M Tokens 32,768 Max Context https://api.fireworks.ai/inference/v1 accounts/fireworks/models/openorca-7b

$0.9/M Tokens 65,536 Max Context https://api.fireworks.ai/inference/v1 accounts/fireworks/models/mixtral-8x22b-instruct-hf

$0.9/M Tokens 65,536 Max Context https://api.fireworks.ai/inference/v1 accounts/fireworks/models/mixtral-8x22b-instruct

$0.5/M Tokens 32,768 Max Context https://api.fireworks.ai/inference/v1 accounts/fireworks/models/mixtral-8x7b-instruct-hf

$0.5/M Tokens 32,768 Max Context https://api.fireworks.ai/inference/v1 accounts/fireworks/models/mixtral-8x7b-instruct

jgravelle commented 4 months ago

Correct on all counts. Groq's paid service will not suffer from this limitation.

In the meantime, you can click any gear icon and change the default model for all agents at once to a non-Groq option...

-jjg

JaloWalt commented 4 months ago

Achieved (almost) what I wanted;

  1. Using the fireworks api in autogroq. Changed the file: AutoGroq/AutoGroq/config.py [attached here as: config-py.txt] config-py.txt

and added the provider file: AutoGroq/AutoGroq/llm_providers/fireworks_provider.py [attached here as: fireworks_provider-py.txt] fireworks_provider-py.txt

  1. Tested that the two above files make fireworks-api to work in AutoGroq. OK, so the test prompt was simple: "Why do dogs bark at cats?" but it still/anyway verifies the basic functionality.

  2. Exported to Autogen. Yes, it did do it.

  3. In autogen; PROBLEMS

Running (in Autogen Playground) the [AutoGroq Workflow] which AutoGroQ exported to Autogen, will produce error.

Reason is following:

The official (and thus correct way) by fireworks.ai is: https://api.fireworks.ai/inference/v1/chat/completions

In conclusion: TODO

A. I will open a bug/error in/for autogen staff to handle.

B. I will use the specific config_local.py described in https://youtu.be/MGEXFuK7RDY (at 1:43+) to fix my (local) config to be resilient regardless AutoGroq updates.

C. Mr J.Gravelle will include the file "fireworks_provider.py" into AutoGroq build.

jgravelle commented 4 months ago

Make sure you share this on Discord, please... -jjg

https://discord.gg/DXjFPX84gs

scruffynerf commented 4 months ago

I mean, you can do openrouter.ai as well: https://openrouter.ai/docs/models

This sort of change should be in the new local config, not in config.py. (making config.py use more custom options like new providers or tweaking existing ones makes sense )

JaloWalt commented 3 months ago

Achieved (almost) what I wanted;

  1. Using the fireworks api in autogroq. Changed the file: AutoGroq/AutoGroq/config.py [attached here as: config-py.txt] config-py.txt

and added the provider file: AutoGroq/AutoGroq/llm_providers/fireworks_provider.py [attached here as: fireworks_provider-py.txt] fireworks_provider-py.txt

  1. Tested that the two above files make fireworks-api to work in AutoGroq. OK, so the test prompt was simple: "Why do dogs bark at cats?" but it still/anyway verifies the basic functionality.
  2. Exported to Autogen. Yes, it did do it.
  3. In autogen; PROBLEMS

Running (in Autogen Playground) the [AutoGroq Workflow] which AutoGroQ exported to Autogen, will produce error.

Reason is following:

The official (and thus correct way) by fireworks.ai is: https://api.fireworks.ai/inference/v1/chat/completions

In conclusion: TODO

A. I will open a bug/error in/for autogen staff to handle.

B. I will use the specific config_local.py described in https://youtu.be/MGEXFuK7RDY (at 1:43+) to fix my (local) config to be resilient regardless AutoGroq updates.

C. Mr J.Gravelle will include the file "fireworks_provider.py" into AutoGroq build.

As regarding the: TODO A. I created the following bug-report to autogen studio;

[Bug]: Autogen Studio App does not use the given baseurl #2903 https://github.com/microsoft/autogen/issues/2903

jgravelle commented 3 months ago

I thought I did 'C.' already...

JaloWalt commented 3 months ago

I thought I did 'C.' already...

Yes, you did. So TODO of C is done.

I will recheck the TODO of A with the newest Autogen Studio. (=0.0.56rc12 of May30th) [How to get it? See: https://github.com/jgravelle/AutoGroq/issues/37]

The TODO of B is of course specific to each user of AutoGroq. I will let you know what I use once I have figured it out myself.

JaloWalt commented 3 months ago

As to: TODO of A.

I updated the autogen-bug-item. BUG still remains.

I have a growing suspicion that autogen will continue making life difficult for everyone else and will (continue) to choose to pick OpenAI-type api-inferences base-urls only if they are given as ending by: "v1"

This means that AutoGroq might need to change any OpenAI-type [NOTE: the TYPE ones and not just actual OpenAI-ones] base-urls to end to "v1" when exporting to autogen. Either that or wait [during the wait having functionality breaking exports] the 2-months release-cycle when MAYBE the autogen has/will? fix this.

jgravelle commented 3 months ago

Agreed. We should wait for both Autogen to finish their re-write and AutoGroq to finish ours...