I'm wondering if it's possible to use custom LLMs f.e. Text-Generation pipelines from huggingface with this package.
What I've tried:
I have used mlflow.transformers.log_model with the task set to: "llm/v1/chat" and then expose the provided model to a databricks serving endpoint.
But this seems not to work since the input schema provided for the mlflow.predict function in langchain-databricks indicates such an error:
"mlflow predict: The params provided to the predict method are not valid"
.
So my question would be:
Is there a working way to deploy a custom llm e.g. from huggingface onto a serving endpoint in order to being able to use langchain-databricks and all its functionalities?
I'm wondering if it's possible to use custom LLMs f.e. Text-Generation pipelines from huggingface with this package.
What I've tried: I have used mlflow.transformers.log_model with the task set to: "llm/v1/chat" and then expose the provided model to a databricks serving endpoint.
But this seems not to work since the input schema provided for the mlflow.predict function in langchain-databricks indicates such an error:
"mlflow predict: The params provided to the predict method are not valid" .
So my question would be:
Is there a working way to deploy a custom llm e.g. from huggingface onto a serving endpoint in order to being able to use langchain-databricks and all its functionalities?
Thanks a lot for your help 😁