API Key not being consumed by langsmith.evaluation.evaluate

AbhilashPal commented 5 months ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

from langsmith.evaluation import evaluate

experiment_results = evaluate(
    langsmith_app, # Your AI system
    data=dataset_name, # The data to predict and grade over
    evaluators=[evaluate_length, qa_evaluator], # The evaluators to score the results
    experiment_prefix="vllm_mistral7b_instruct_", # A prefix for your experiment names to easily identify them
    client=client,
)

Error Message and Stack Trace (if applicable)

Traceback (most recent call last):
  File "/opt/conda/lib/python3.11/site-packages/langsmith/evaluation/_runner.py", line 1231, in _run_evaluators
    evaluator_response = evaluator.evaluate_run(
                         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/langsmith/evaluation/evaluator.py", line 279, in evaluate_run
    result = self.func(
             ^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/langsmith/run_helpers.py", line 546, in wrapper
    run_container = _setup_run(
                    ^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/langsmith/run_helpers.py", line 1078, in _setup_run
    new_run = run_trees.RunTree(
              ^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/pydantic/v1/main.py", line 339, in __init__
    values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/pydantic/v1/main.py", line 1074, in validate_model
    v_, errors_ = field.validate(value, values, loc=field.alias, cls=cls_)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/pydantic/v1/fields.py", line 864, in validate
    v, errors = self._apply_validators(v, values, loc, cls, self.pre_validators)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/pydantic/v1/fields.py", line 1154, in _apply_validators
    v = validator(cls, v, values, self, self.model_config)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/pydantic/v1/class_validators.py", line 304, in <lambda>
    return lambda cls, v, values, field, config: validator(cls, v)
                                                 ^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/langsmith/run_trees.py", line 63, in validate_client
    return Client()
           ^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/langsmith/client.py", line 534, in __init__
    _validate_api_key_if_hosted(self.api_url, self.api_key)
  File "/opt/conda/lib/python3.11/site-packages/langsmith/client.py", line 323, in _validate_api_key_if_hosted
    raise ls_utils.LangSmithUserError(
langsmith.utils.LangSmithUserError: API key must be provided when using hosted LangSmith API

Description

I have input my api_key in the langsmith client, furthermore I have also exported the LANGCHAIN_API_KEY as well. However, when I try to run the code.

System Info

langchain==0.2.1 langchain-anthropic==0.1.13 langchain-core==0.2.1 langchain-openai==0.1.7 langchain-text-splitters==0.2.0

shobhitagnihotri69 commented 5 months ago

Yes some time .env file does not work in langsmith . I have encountered same try using LANGCHAIN_API_KEY = "", LANGCHAIN_TRACING_V2 = "true"

hinthornw commented 5 months ago

Could you clarify what y ou mean by .env file not working with langsmith?

it has to be loaded beforehand, but i'm guessing you're already doing that

Lukecn1 commented 2 months ago

I am encountering the same problem - did you find a fix for this?

langchain-ai / langchain