Issue: custom metadata sent to `RemoteRunnable(...).with_config(metadata={...}).{invoke,stream,etc}()` is not provided to the runnable in langserve.api_handler.APIHandler._get_config_and_input #694
Via discord thread in langsmith-support-forumhere:
I have both the context that calls RemoteRunnable(...).with_config(metadata={"my_key": "my_val"}).invoke(...) and the LangServe API target running the LCEL runnable correctly wired up with langsmith; all runs are getting recorded by both. I'm observing that the metadata I put in gets recorded to the RemoteRunnable run, but not to the api invocation run on the LangServe server. Do I need to set up my add_routes some way to pass through additional metadata?
I can see that when I drop a breakpoint in langserve.api_handler.APIHandler._get_config_and_input inside of invoke(), the config that's passed here does have a metadata key with some auto-added keys from langserve...
but my custom key is not here anymore.
I can verify that (await request.json())['config']['metadata']['my_key'] shows the correct value in that method, though.
To my eyes, it looks like the _unpack_request_config called in that method returns a user_provided_config that only contains the configuration, not the metadata I supplied.
Expected Behavior
If metadata is supplied to RemoteRunnable(...).with_config(metadata={...}).{invoke,stream,etc}(), then the LangServe endpoint should supply that metadata to the configuration of the Runnable before running it.
Actual Behavior
If metadata is supplied to RemoteRunnable(...).with_config(metadata={...}).{invoke,stream,etc}(), then the LangServe endpoint is not suppling that metadata to the configuration of the Runnable before running it.
Other information
I'm on LangSmith Plus organization with 7+ seats
langserve version: 0.2.2
Suggestion:
Please ensure metadata passed to RemoteRunnable is added to the configuration of the runnable on the server before invoking the Runnable.
Via discord thread in
langsmith-support-forum
here:Expected Behavior
If
metadata
is supplied toRemoteRunnable(...).with_config(metadata={...}).{invoke,stream,etc}()
, then the LangServe endpoint should supply that metadata to the configuration of theRunnable
before running it.Actual Behavior
If
metadata
is supplied toRemoteRunnable(...).with_config(metadata={...}).{invoke,stream,etc}()
, then the LangServe endpoint is not suppling that metadata to the configuration of theRunnable
before running it.Other information
I'm on LangSmith Plus organization with 7+ seats
langserve
version:0.2.2
Suggestion:
Please ensure metadata passed to
RemoteRunnable
is added to the configuration of the runnable on the server before invoking the Runnable.