Issue: Add exception handling for nonexisting spits

[jiayini1119]

Issue you'd like to raise.

Hi LangChain Community,

I discovered that if I do

from langsmith import Client
from langsmith.evaluation import evaluate
client = Client()

and pass in data = client.list_examples(dataset_name = dataset_name, splits=[{"existing_split", "nonexisting_split"}]) ("existing_split" is a split that exists while "nonexisting_split" is a split that does not exist) in LangSmith's evaluate function, LangSmith will ignore the "nonexisting_split" and only evaluate on the "existing_split".

It would be better if error is raised for this case.

Thanks!

Suggestion:

No response

[hinthornw]

So the request here is to raise an error in in the examples GET endpoint if a split that is provided isn't found in the dataset?

[jiayini1119]

Yes.