Closed stargazing-dino closed 1 month ago
@stargazing-dino First, can you squash those commits into one, so that it's faster for me to review? Perhaps creating a new branch and doing a fresh PR with the squashed commits might be good.
Second, I need to think about this. I do not guarantee I will take it as is.
I will look at the code, but there are other dimensions. For example, the API Key might be more by AdapterKind, and then I also need to add a custom endpoint. So, I need to think about all of this together. But your commit can help me clarify one requirement at least.
Thanks.
@stargazing-dino Sorry, this does not really fit the API model in mind. I do think it is important to split the model and even adapters from the client.
One way to accomplish this behavior would be to implement an application struct that would have the client, the appropriate default, and re-expose the exec in a custom way.
I will close this PR.
By the way, there are two features I am going to implement that will allow you to accomplish what you described.
1) Model alias - the Client/ClientConfig will have aliases:
client_builder
.append_alias("big-one", "gpt-4o")
.append_alias("small-one", "llama-3.1-70b-versatile")
So, the developer can also set an alias for "" and that will work as well.
2) AuthResolver refactor to be at the Client level - Now that we have ModelInfo
with adapter_kind
and model_name
, I am going to refactor the AuthResolver to be at the ClientConfig level (so, no need for AdapterConfig for now and probably later as well). This will simplify the API and implementation a lot.
let client = Client::builder()
.with_sync_auth_resolver(|model_info: ModelInfo| { ... })
.build();
This new AuthResolver model is much simpler, same flexibility, one entry point for all.
Allows something like this:
so users don't have to pass it at every call site of
exec_chat