langchain-ai / langsmith-sdk

LangSmith Client SDK Implementations
https://smith.langchain.com/
MIT License
370 stars 66 forks source link

Issue: No longer able to use the `run_ids` argument of the `list_runs` function #859

Closed tituskx closed 1 day ago

tituskx commented 1 month ago

Issue you'd like to raise.

I am no longer able to retrieve runs by using the run_ids argument of the list_runs function. The following example

    runs = list(
        client.list_runs(
            project_name=project_name,
            start_time=start_time,
            end_time=end_time,
            filter=f"and(eq(metadata_key, 'organisation'), eq(metadata_value, '{organisation}'))",
            is_root=True,
        )
    )
    child_run_ids = [
        child_run_id
        for run in runs[:num_root_runs]
        for child_run_id in run.child_run_ids
    ]
    child_runs = list(
        client.list_runs(
            project_name=project_name,
            child_run_ids=child_run_ids,
        )
    )

raises the following error

Server error caused failure to POST /runs/query in LangSmith API. HTTPError('500 Server Error: Internal Server Error for url: https://api.smith.langchain.com/runs/query', '{"detail":"Internal server error"}') requests.exceptions.HTTPError: 500 Server Error: Internal Server Error for url: https://api.smith.langchain.com/runs/query

This worked this morning. If I execute the above code the part

    child_runs = list(
        client.list_runs(
            project_name=project_name,
            child_run_ids=child_run_ids,
        )
    )

raises the 500 exception. The following part runs fine

    runs = list(
        client.list_runs(
            project_name=project_name,
            start_time=start_time,
            end_time=end_time,
            filter=f"and(eq(metadata_key, 'organisation'), eq(metadata_value, '{organisation}'))",
            is_root=True,
        )
    )

Suggestion:

No response

hinthornw commented 1 month ago

Hm. Your code runs fine for me so long as the "child_run_ids" is changed to "run_ids"

child_runs = list(
        client.list_runs(
            project_name=project_name,
            run_ids=child_run_ids,
        )
    )

Will flag this to the infra folks, since obviously a 500 error is an issue.

tituskx commented 1 month ago

@hinthornw Awesome thanks! Sorry about the child_run_ids argument that was a typo:)