Fix Replicate API Key Passing

[fayvor]

langchain-community.llms.replicate.Replicate doesn't pass its replicate_api_key parameter to the client it uses to access the service to get the version_obj and other things in _create_prediction().

Step 1:

• Fix the new version of Replicate and get the tests to pass (these are both broken).

Step 2:

• Fix the api key passthrough. One way to do this is by creating a Client internal to the Replicate object to use for operations that require service access.

[fayvor]

Here is the call stack to the unauthenticated request. The client it is using is the default Client, and it has no api_key.

_create_prediction (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_community/llms/replicate.py:204)
_call (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_community/llms/replicate.py:139)
_generate (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1508)
_generate_helper (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_core/language_models/llms.py:774)
generate (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_core/language_models/llms.py:944)
<module> (/Users/fayvor/Dev/granite-kitchen/replicate_client.py:16)
_run_code (/Users/fayvor/.pyenv/versions/3.10.15/lib/python3.10/runpy.py:86)
_run_module_as_main (/Users/fayvor/.pyenv/versions/3.10.15/lib/python3.10/runpy.py:196)
get (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/replicate/model.py:264)
_create_prediction (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_community/llms/replicate.py:204)
_call (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_community/llms/replicate.py:139)
_generate (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1508)
_generate_helper (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_core/language_models/llms.py:774)
generate (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_core/language_models/llms.py:944)
<module> (/Users/fayvor/Dev/granite-kitchen/replicate_client.py:16)
_run_code (/Users/fayvor/.pyenv/versions/3.10.15/lib/python3.10/runpy.py:86)
_run_module_as_main (/Users/fayvor/.pyenv/versions/3.10.15/lib/python3.10/runpy.py:196)```

[fayvor]

Default client is instantiated here: https://github.com/replicate/replicate-python/blob/main/replicate/__init__.py#L5

When we are in this stack:

<module> (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/replicate/__init__.py:5)
_create_prediction (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_community/llms/replicate.py:190)
_call (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_community/llms/replicate.py:139)
_generate (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1508)
_generate_helper (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_core/language_models/llms.py:774)
generate (/Users/fayvor/Dev/granite-kitchen/.venv/lib/python3.10/site-packages/langchain_core/language_models/llms.py:944)
<module> (/Users/fayvor/Dev/granite-kitchen/replicate_client.py:16)
_run_code (/Users/fayvor/.pyenv/versions/3.10.15/lib/python3.10/runpy.py:86)
_run_module_as_main (/Users/fayvor/.pyenv/versions/3.10.15/lib/python3.10/runpy.py:196)```

[fayvor]

This is where the REPLICATE_API_TOKEN is getting set in the httpx client: https://github.com/replicate/replicate-python/blob/main/replicate/client.py#L362

[fayvor]

I was able to get this to work by setting the api token directly on the replicate module's default_client:

import replicate
replicate.default_client._api_token = <token>

I wouldn't use this in recipes since it accesses a "private" variable, but it demonstrates what is needed. The replicate.default_client is used for things like retrieving model info from the service.

[fayvor]

Langchain PR here to fix the most recent version of the Replicate LLM class. - MERGED

I can probably now build off of this to fix the api key passthrough.

NOTE: If no one reviews your PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.

[fayvor]

This PR fixes the Replicate API key passing.

NOTE: If no one reviews the PR within a few days, please @-mention one of baskaryan, efriis, eyurtsev, ccurme, vbarda, hwchase17.