default_client is ignored if model is in default registry

[ppabis]

This comment has the latest update: https://github.com/MadcowD/ell/issues/366#issuecomment-2466455721

Anything below is likely not an issue.

So, when setting custom client, both in ell.init and decorator, the change is not always reflected in ell internals. The problem returns at "random" point (because it always happens with anthropic.claude-3-haiku-20240307-v1:0 and not any other model ID), probably when I set bedrock client and use other model that uses this client.

The error I get (when explicitly setting bedrock as the client in both decorator and init):

    raise RuntimeError(_no_api_key_warning(model, _name, '', long=True, error=True))
RuntimeError: ERROR: No API key found for model `anthropic.claude-3-haiku-20240307-v1:0` using client `OpenAI`.

To fix this:
* Set your API key in the appropriate environment variable for your chosen provider
...

However, when I create a print statement in src/ell/lmp/complex.py: https://github.com/MadcowD/ell/blob/3ce6a21d4bdf64802e4a09368d86d9abc204a9c4/src/ell/lmp/complex.py#L53

            merged_api_params = {**config.default_api_params, **default_api_params_from_decorator, **(api_params or {})}
            n = merged_api_params.get("n", 1)
            # Merge client overrides & client registry
            merged_client = _client_for_model(model, client or default_client_from_decorator)
            print(merged_client) # Added 
            ell_call = EllCallParams(

Everything works and I see the object printed

<botocore.client.BedrockRuntime object at 0x110396510>

$ python -VV # on macOS
Python 3.12.4 (main, Jun  6 2024, 18:26:44) [Clang 15.0.0 (clang-1500.3.9.4)]

[ppabis]

I tested it in another code and it happens only with anthropic.claude-3-haiku-20240307-v1:0, any other model ID did not cause this issues, including anthropic.claude-3-5-haiku-20241022-v1:0. I don't know if it is some pattern matching or unfortunate hash somewhere deep inside Python.

[ppabis]

Ok, I figured out something - it happens only with anthropic.claude-3-haiku-20240307-v1:0 and it's the only known model in ell package. It points to bedrock. However, it forces None value on the client and disregards default_client when using ell.init, thus trying OpenAI one.

[alex-dixon]

@ppabis This isn’t really a solution but if it helps unstick you until we can fix whatever’s happening you can try supplying the client argument to the decorator directly

I realized you mentioned you tried this but I would expect it to work if it’s on every decorator.

Are you doing any multithreading or multiprocessing or anything?