search

googleapis / python-aiplatform

A Python SDK for Vertex AI, a fully managed, end-to-end platform for data science and machine learning.
Apache License 2.0
653 stars 348 forks source link

Batch predict using custom models with mandatory parameter fields is not supported in vertex AI #2666

Open anuragbisht-turing opened 1 year ago

anuragbisht-turing commented 1 year ago

We have custom models that requires passing mandatory parameter field for prediction. Now this works for online predictions via endpoint in POST request, but for batch predictions we have to create jsonl file . The format of each line in jsonl is something like this 

{"id1":1,"id2":1,"timestamp":"2023-09-28 09:55:47Z","key1":{some dict},"key2":{some dict}}
{...}

etc

This throws below error as parameter field is not present.

We tried adding it too but looks it doesn't work either. Can someone let us know to make it work as I see this as a bug.

Environment details

Steps to reproduce

  1. deploy a custom model with mandatory parameter in the request body for prediction
  2. create batch input jsonl file in gcs bucket , use this file as input to batch predict method of the model.
  3. Notice the job completion output error in the gcs bucket.

Code example

job = model.batch_predict(machine_type="n1-standard-16", max_replica_count=3, job_display_name="test_score", gcs_source="gs://test/batch_predict_requests.jsonl", batch_size=1000, model_parameters=body["parameters"], gcs_destination_prefix="gs://test/output_dir", sync=True)

Stack trace

('Post request fails. Cannot get predictions. Error: Exceeded retries: Non-OK result 422 ({"detail":[{"loc":["body","parameters"],"msg":"field required","type":"value_error.missing"}]}) from server, retry=3, ellapsed=0.13s.', 1000)

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

ggianotticonsello commented 8 months ago

The same is happening to me