search

aws / amazon-sagemaker-examples

Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
https://sagemaker-examples.readthedocs.io
Apache License 2.0
10.14k stars 6.78k forks source link

bandits_statlog_vw_customEnv: Customer Error: Unable to load model (caused by JSONDecodeError) #1304

Open tvkpz opened 4 years ago

tvkpz commented 4 years ago

The bandits statlog Vowpal wabbit example is not able to load the model at the step

bandits_experiment.deploy_model(model_id=bandits_experiment.last_trained_model_id)

Here is trace:

algo-1-yklna_1 | Caused by: Expecting value: line 1 column 1 (char 0) algo-1-yklna_1 | Traceback (most recent call last): algo-1-yklna_1 | File "/usr/local/lib/python3.6/dist-packages/vw_serving/serve.py", line 235, in get_model algo-1-yklna_1 | quiet_mode=True) algo-1-yklna_1 | File "/usr/local/lib/python3.6/dist-packages/vw_serving/vw_agent.py", line 289, in load_model algo-1-yklna_1 | metadata = json.loads(metadata) algo-1-yklna_1 | File "/usr/lib/python3.6/json/init.py", line 354, in loads algo-1-yklna_1 | return _default_decoder.decode(s) algo-1-yklna_1 | File "/usr/lib/python3.6/json/decoder.py", line 339, in decode algo-1-yklna_1 | obj, end = self.raw_decode(s, idx=_w(s, 0).end()) algo-1-yklna_1 | File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode algo-1-yklna_1 | raise JSONDecodeError("Expecting value", s, err.value) from None algo-1-yklna_1 | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

annaluo676 commented 4 years ago

Sorry for the delayed response. From the logs it seems the vw agent is not ready. Can you retry the example, make sure the last retraining is finished and ready for deployment?