Open awolf0 opened 1 month ago
Thanks! Could you file a PR and attach testing result ? I don't have Bedrock account token to test it.
CC @andrew-christianson Would you help checking this ? Thx!
@mlflow/mlflow-team Please assign a maintainer and start triaging this issue.
Hi guys, any updates on this issue or comments on the PR? This is my first contribution to mlflow, so please treat it as such, and I would appreciate any guidance :-)
I am getting the same sort of issue when using claude-3-5-sonnet-20240620
. When I try to create a new run with this endpoint I get:
MLflow deployment returned the following error: "Deployments proxy request failed with error code 400. Error message: {"detail":"\"claude-3-5-sonnet-20240620\" is not supported on this API. Please use the Messages API instead."}"
Is there any work-around in the meantime so that I can still use Claude 3.5 Sonnet with MLFlow?
Issues Policy acknowledgement
Where did you encounter this bug?
Other
Willingness to contribute
Yes. I would be willing to contribute a fix for this bug with guidance from the MLflow community.
MLflow version
System information
Describe the problem
I have set up two endpoints for Anthropic Claude 3 Sonnet on Bedrock, one for completions, and one for chat. Sending a request to the completions endpoint returns
"claude-3-sonnet-20240229" is not supported on this API. Please use the Messages API instead.
Sending a request to the chat endpoint returnsThe chat route is not implemented for Amazon Bedrock models.
.Tracking information
Code to reproduce issue
configuration:
Stack trace
The response for the request to chat:
Other info / logs
What component(s) does this bug affect?
area/artifacts
: Artifact stores and artifact loggingarea/build
: Build and test infrastructure for MLflowarea/deployments
: MLflow Deployments client APIs, server, and third-party Deployments integrationsarea/docs
: MLflow documentation pagesarea/examples
: Example codearea/model-registry
: Model Registry service, APIs, and the fluent client calls for Model Registryarea/models
: MLmodel format, model serialization/deserialization, flavorsarea/recipes
: Recipes, Recipe APIs, Recipe configs, Recipe Templatesarea/projects
: MLproject format, project running backendsarea/scoring
: MLflow Model server, model deployment tools, Spark UDFsarea/server-infra
: MLflow Tracking server backendarea/tracking
: Tracking Service, tracking client APIs, autologgingWhat interface(s) does this bug affect?
area/uiux
: Front-end, user experience, plotting, JavaScript, JavaScript dev serverarea/docker
: Docker use across MLflow's components, such as MLflow Projects and MLflow Modelsarea/sqlalchemy
: Use of SQLAlchemy in the Tracking Service or Model Registryarea/windows
: Windows supportWhat language(s) does this bug affect?
language/r
: R APIs and clientslanguage/java
: Java APIs and clientslanguage/new
: Proposals for new client languagesWhat integration(s) does this bug affect?
integrations/azure
: Azure and Azure ML integrationsintegrations/sagemaker
: SageMaker integrationsintegrations/databricks
: Databricks integrations