Describe the bug
I'm trying to create a model version with the custom prediction routine beta feature. The create keeps failing with an Unexpected error. I've tried creating the version through iTerm and in the AI Platform UI.
Source code / logsCreate Version failed. Bad model detected with error: "Failed to load model: Unexpected error when loading the model: 0 (Error code: 0)"
Full traceback:
DEBUG: Running [gcloud.beta.ai-platform.versions.create] with arguments: [--model: "test_model", --origin: "gs://test_model/models/", --package-uris: "[u'gs://test_model/models/my_custom_code-0.1.tar.gz']", --prediction-class: "predictor.MyPredictor", --python-version: "3.5", --runtime-version: "1.13", --verbosity: "debug", VERSION: "version_test"]
Creating version (this might take a few minutes)......failed.
DEBUG: (gcloud.beta.ai-platform.versions.create) Create Version failed. Bad model detected with error: "Failed to load model: Unexpected error when loading the model: 9 (Error code: 0)"
Traceback (most recent call last):
File "/Users/user_1/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 985, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "/Users/user_1/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 795, in Run
resources = command_instance.Run(args)
File "/Users/user_1/google-cloud-sdk/lib/surface/ai_platform/versions/create.py", line 158, in Run
package_uris=args.package_uris)
File "/Users/user_1/google-cloud-sdk/lib/googlecloudsdk/command_lib/ml_engine/versions_util.py", line 113, in Create
message='Creating version (this might take a few minutes)...')
File "/Users/user_1/google-cloud-sdk/lib/googlecloudsdk/command_lib/ml_engine/versions_util.py", line 74, in WaitForOpMaybe
return operations_client.WaitForOperation(op, message=message).response
File "/Users/user_1/google-cloud-sdk/lib/googlecloudsdk/api_lib/ml_engine/operations.py", line 114, in WaitForOperation
sleep_ms=5000)
File "/Users/user_1/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 264, in WaitFor
sleep_ms, _StatusUpdate)
File "/Users/user_1/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 326, in PollUntilDone
sleep_ms=sleep_ms)
File "/Users/user_1/google-cloud-sdk/lib/googlecloudsdk/core/util/retry.py", line 229, in RetryOnResult
if not should_retry(result, state):
File "/Users/user_1/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 320, in _IsNotDone
return not poller.IsDone(operation)
File "/Users/user_1/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 122, in IsDone
raise OperationError(operation.error.message)
OperationError: Create Version failed. Bad model detected with error: "Failed to load model: Unexpected error when loading the model: 0 (Error code: 0)"
ERROR: (gcloud.beta.ai-platform.versions.create) Create Version failed. Bad model detected with error: "Failed to load model: Unexpected error when loading the model: 9 (Error code: 0)"
System Information
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS Mojave 10.14.4
Framework and version (Tensorflow, scikit-learn, XGBoost): scikit-learn
I suspect I might need to include a preprocessor in my custom routine as well (I only need a custom predictor so didn't include a preprocessor from the start).
Describe the bug I'm trying to create a model version with the custom prediction routine beta feature. The create keeps failing with an
Unexpected error
. I've tried creating the version through iTerm and in the AI Platform UI.What sample is this bug related to? https://cloud.google.com/ml-engine/docs/scikit/custom-prediction-routine-scikit-learn
Source code / logs
Create Version failed. Bad model detected with error: "Failed to load model: Unexpected error when loading the model: 0 (Error code: 0)"
Full traceback:
System Information
Additional context I've tried updating
sklearn
to version0.20.2
after finding this issue with no success: https://github.com/GoogleCloudPlatform/cloudml-samples/issues/419I suspect I might need to include a preprocessor in my custom routine as well (I only need a custom predictor so didn't include a preprocessor from the start).