xgboost example fails

[pablocastilla]

Hi!

I am trying these examples. I am interested in xgboost but it seems to fail. We user using windows 11, python 3.7.9, vs code and docker. The downloaded image is 141502667606.DKR.ECR.EU-WEST-1.AMAZONAWS.COM/SAGEMAKER-XGBOOST

The exception is the following:

f3k8ih5gq0-algo-1-9che5 | self._model = self._model_fn(_env.model_dir) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 95, in wrapper f3k8ih5gq0-algo-1-9che5 | six.reraise(error_class, error_class(e), sys.exc_info()[2]) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/six.py", line 718, in reraise f3k8ih5gq0-algo-1-9che5 | raise value.with_traceback(tb) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper f3k8ih5gq0-algo-1-9che5 | return fn(*args, kwargs) f3k8ih5gq0-algo-1-9che5 | File "/opt/ml/code/inference.py", line 26, in model_fn f3k8ih5gq0-algo-1-9che5 | booster = pkl.load(open(os.path.join(model_dir, model_file), "rb")) f3k8ih5gq0-algo-1-9che5 | sagemaker_containers._errors.ClientError: [Errno 2] No such file or directory: '/opt/ml/model/xgboost-model' f3k8ih5gq0-algo-1-9che5 | 172.18.0.1 - - [23/May/2022:09:31:41 +0000] "GET /ping HTTP/1.1" 500 141 "-" "python-urllib3/1.26.9" f3k8ih5gq0-algo-1-9che5 | [2022-05-23:09:31:46:INFO] No GPUs detected (normal if no gpus installed) f3k8ih5gq0-algo-1-9che5 | [2022-05-23:09:31:46:INFO] Installing module with the following command: f3k8ih5gq0-algo-1-9che5 | /miniconda3/bin/python3 -m pip install . f3k8ih5gq0-algo-1-9che5 | Processing /opt/ml/code f3k8ih5gq0-algo-1-9che5 | Preparing metadata (setup.py) ... done f3k8ih5gq0-algo-1-9che5 | Building wheels for collected packages: inference f3k8ih5gq0-algo-1-9che5 | Building wheel for inference (setup.py) ... done f3k8ih5gq0-algo-1-9che5 | Created wheel for inference: filename=inference-1.0.0-py2.py3-none-any.whl size=13189 sha256=0343ebd456f9251526f64d5c6ae851b88410b3b053b71c21e41a94726ad2b5a2 f3k8ih5gq0-algo-1-9che5 | Stored in directory: /home/model-server/tmp/pip-ephem-wheel-cache-bs58b9ud/wheels/3e/0f/51/2f1df833dd0412c1bc2f5ee56baac195b5be563353d111dca6 f3k8ih5gq0-algo-1-9che5 | Successfully built inference f3k8ih5gq0-algo-1-9che5 | Installing collected packages: inference f3k8ih5gq0-algo-1-9che5 | Attempting uninstall: inference f3k8ih5gq0-algo-1-9che5 | Found existing installation: inference 1.0.0 f3k8ih5gq0-algo-1-9che5 | Can't uninstall 'inference'. No files were found to uninstall. f3k8ih5gq0-algo-1-9che5 | Successfully installed inference-1.0.0 f3k8ih5gq0-algo-1-9che5 | WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv f3k8ih5gq0-algo-1-9che5 | WARNING: You are using pip version 22.0.4; however, version 22.1.1 is available. f3k8ih5gq0-algo-1-9che5 | You should consider upgrading via the '/miniconda3/bin/python3 -m pip install --upgrade pip' command. f3k8ih5gq0-algo-1-9che5 | [2022-05-23 09:31:47 +0000] [225] [ERROR] Error handling request /ping f3k8ih5gq0-algo-1-9che5 | Traceback (most recent call last): f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper f3k8ih5gq0-algo-1-9che5 | return fn(*args, *kwargs) f3k8ih5gq0-algo-1-9che5 | File "/opt/ml/code/inference.py", line 26, in model_fn f3k8ih5gq0-algo-1-9che5 | booster = pkl.load(open(os.path.join(model_dir, model_file), "rb")) f3k8ih5gq0-algo-1-9che5 | FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/model/xgboost-model' f3k8ih5gq0-algo-1-9che5 | f3k8ih5gq0-algo-1-9che5 | During handling of the above exception, another exception occurred: f3k8ih5gq0-algo-1-9che5 | f3k8ih5gq0-algo-1-9che5 | Traceback (most recent call last): f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 56, in handle f3k8ih5gq0-algo-1-9che5 | self.handle_request(listener_name, req, client, addr) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/gunicorn/workers/ggevent.py", line 160, in handle_request f3k8ih5gq0-algo-1-9che5 | addr) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 107, in handle_request f3k8ih5gq0-algo-1-9che5 | respiter = self.wsgi(environ, resp.start_response) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_xgboost_container/serving.py", line 136, in main f3k8ih5gq0-algo-1-9che5 | user_module_transformer.initialize() f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_transformer.py", line 185, in initialize f3k8ih5gq0-algo-1-9che5 | self._model = self._model_fn(_env.model_dir) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 95, in wrapper f3k8ih5gq0-algo-1-9che5 | six.reraise(error_class, error_class(e), sys.exc_info()[2]) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/six.py", line 718, in reraise f3k8ih5gq0-algo-1-9che5 | raise value.with_traceback(tb) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper f3k8ih5gq0-algo-1-9che5 | return fn(args, kwargs) f3k8ih5gq0-algo-1-9che5 | File "/opt/ml/code/inference.py", line 26, in model_fn f3k8ih5gq0-algo-1-9che5 | booster = pkl.load(open(os.path.join(model_dir, model_file), "rb")) f3k8ih5gq0-algo-1-9che5 | sagemaker_containers._errors.ClientError: [Errno 2] No such file or directory: '/opt/ml/model/xgboost-model' f3k8ih5gq0-algo-1-9che5 | 172.18.0.1 - - [23/May/2022:09:31:47 +0000] "GET /ping HTTP/1.1" 500 141 "-" "python-urllib3/1.26.9" f3k8ih5gq0-algo-1-9che5 | [2022-05-23:09:31:52:INFO] No GPUs detected (normal if no gpus installed) f3k8ih5gq0-algo-1-9che5 | [2022-05-23:09:31:52:INFO] Installing module with the following command: f3k8ih5gq0-algo-1-9che5 | /miniconda3/bin/python3 -m pip install . f3k8ih5gq0-algo-1-9che5 | Processing /opt/ml/code f3k8ih5gq0-algo-1-9che5 | Preparing metadata (setup.py) ... done f3k8ih5gq0-algo-1-9che5 | Building wheels for collected packages: inference f3k8ih5gq0-algo-1-9che5 | Building wheel for inference (setup.py) ... done f3k8ih5gq0-algo-1-9che5 | Created wheel for inference: filename=inference-1.0.0-py2.py3-none-any.whl size=16359 sha256=56f1e8fab2fec8c4a15850ac4ecf9a02b73f0068434821b09cf73cbcda092824 f3k8ih5gq0-algo-1-9che5 | Stored in directory: /home/model-server/tmp/pip-ephem-wheel-cache-02pq641a/wheels/3e/0f/51/2f1df833dd0412c1bc2f5ee56baac195b5be563353d111dca6 f3k8ih5gq0-algo-1-9che5 | Successfully built inference f3k8ih5gq0-algo-1-9che5 | Installing collected packages: inference f3k8ih5gq0-algo-1-9che5 | Attempting uninstall: inference f3k8ih5gq0-algo-1-9che5 | Found existing installation: inference 1.0.0 f3k8ih5gq0-algo-1-9che5 | Can't uninstall 'inference'. No files were found to uninstall. f3k8ih5gq0-algo-1-9che5 | Successfully installed inference-1.0.0 f3k8ih5gq0-algo-1-9che5 | WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv f3k8ih5gq0-algo-1-9che5 | WARNING: You are using pip version 22.0.4; however, version 22.1.1 is available. f3k8ih5gq0-algo-1-9che5 | You should consider upgrading via the '/miniconda3/bin/python3 -m pip install --upgrade pip' command. f3k8ih5gq0-algo-1-9che5 | [2022-05-23 09:31:53 +0000] [227] [ERROR] Error handling request /ping f3k8ih5gq0-algo-1-9che5 | Traceback (most recent call last): f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper f3k8ih5gq0-algo-1-9che5 | return fn(*args, kwargs) f3k8ih5gq0-algo-1-9che5 | File "/opt/ml/code/inference.py", line 26, in model_fn f3k8ih5gq0-algo-1-9che5 | booster = pkl.load(open(os.path.join(model_dir, model_file), "rb")) f3k8ih5gq0-algo-1-9che5 | FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/model/xgboost-model' f3k8ih5gq0-algo-1-9che5 | f3k8ih5gq0-algo-1-9che5 | During handling of the above exception, another exception occurred: f3k8ih5gq0-algo-1-9che5 | f3k8ih5gq0-algo-1-9che5 | Traceback (most recent call last): f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 56, in handle f3k8ih5gq0-algo-1-9che5 | self.handle_request(listener_name, req, client, addr) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/gunicorn/workers/ggevent.py", line 160, in handle_request f3k8ih5gq0-algo-1-9che5 | addr) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 107, in handle_request f3k8ih5gq0-algo-1-9che5 | respiter = self.wsgi(environ, resp.start_response) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_xgboost_container/serving.py", line 136, in main f3k8ih5gq0-algo-1-9che5 | user_module_transformer.initialize() f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_transformer.py", line 185, in initialize f3k8ih5gq0-algo-1-9che5 | self._model = self._model_fn(_env.model_dir) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 95, in wrapper f3k8ih5gq0-algo-1-9che5 | six.reraise(error_class, error_class(e), sys.exc_info()[2]) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/six.py", line 718, in reraise f3k8ih5gq0-algo-1-9che5 | raise value.with_traceback(tb) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper f3k8ih5gq0-algo-1-9che5 | return fn(*args, *kwargs) f3k8ih5gq0-algo-1-9che5 | File "/opt/ml/code/inference.py", line 26, in model_fn f3k8ih5gq0-algo-1-9che5 | booster = pkl.load(open(os.path.join(model_dir, model_file), "rb")) f3k8ih5gq0-algo-1-9che5 | sagemaker_containers._errors.ClientError: [Errno 2] No such file or directory: '/opt/ml/model/xgboost-model' f3k8ih5gq0-algo-1-9che5 | 172.18.0.1 - - [23/May/2022:09:31:53 +0000] "GET /ping HTTP/1.1" 500 141 "-" "python-urllib3/1.26.9" f3k8ih5gq0-algo-1-9che5 | [2022-05-23:09:31:58:INFO] No GPUs detected (normal if no gpus installed) f3k8ih5gq0-algo-1-9che5 | [2022-05-23:09:31:58:INFO] Installing module with the following command: f3k8ih5gq0-algo-1-9che5 | /miniconda3/bin/python3 -m pip install . f3k8ih5gq0-algo-1-9che5 | Processing /opt/ml/code f3k8ih5gq0-algo-1-9che5 | Preparing metadata (setup.py) ... done f3k8ih5gq0-algo-1-9che5 | Building wheels for collected packages: inference f3k8ih5gq0-algo-1-9che5 | Building wheel for inference (setup.py) ... done f3k8ih5gq0-algo-1-9che5 | Created wheel for inference: filename=inference-1.0.0-py2.py3-none-any.whl size=19706 sha256=4e5bdcd2e59629fadf0c0e54a3d71f769dd92d167d08729e7ab0f4b9074a7066 f3k8ih5gq0-algo-1-9che5 | Stored in directory: /home/model-server/tmp/pip-ephem-wheel-cache-vq93idru/wheels/3e/0f/51/2f1df833dd0412c1bc2f5ee56baac195b5be563353d111dca6 f3k8ih5gq0-algo-1-9che5 | Successfully built inference f3k8ih5gq0-algo-1-9che5 | Installing collected packages: inference f3k8ih5gq0-algo-1-9che5 | Attempting uninstall: inference f3k8ih5gq0-algo-1-9che5 | Found existing installation: inference 1.0.0 f3k8ih5gq0-algo-1-9che5 | Can't uninstall 'inference'. No files were found to uninstall. f3k8ih5gq0-algo-1-9che5 | Successfully installed inference-1.0.0 f3k8ih5gq0-algo-1-9che5 | WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv f3k8ih5gq0-algo-1-9che5 | WARNING: You are using pip version 22.0.4; however, version 22.1.1 is available. f3k8ih5gq0-algo-1-9che5 | You should consider upgrading via the '/miniconda3/bin/python3 -m pip install --upgrade pip' command. f3k8ih5gq0-algo-1-9che5 | [2022-05-23 09:31:59 +0000] [225] [ERROR] Error handling request /ping f3k8ih5gq0-algo-1-9che5 | Traceback (most recent call last): f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper f3k8ih5gq0-algo-1-9che5 | return fn(args, kwargs) f3k8ih5gq0-algo-1-9che5 | File "/opt/ml/code/inference.py", line 26, in model_fn f3k8ih5gq0-algo-1-9che5 | booster = pkl.load(open(os.path.join(model_dir, model_file), "rb")) f3k8ih5gq0-algo-1-9che5 | FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/model/xgboost-model'

Am I doing something wrong?

[eitansela]

Hi,

The key error is: "No such file or directory: '/opt/ml/model/xgboost-model '. This means that *inference.py* tried to load a model file named *xgboost-model*, and this file is taken from model.tar.gz in S3 during model deployment.

For debugging purposes, can you print the content of */opt/ml/model/* folder?

On Mon, May 23, 2022 at 11:49 AM Pablo Castilla @.***> wrote:

Hi!

I am trying these examples. I am interested in xgboost but it seems to fail. We user using windows 11, python 3.7.9, vs code and docker. The downloaded image is 141502667606.DKR.ECR.EU-WEST-1.AMAZONAWS.COM/SAGEMAKER-XGBOOST

The exception is the following:

f3k8ih5gq0-algo-1-9che5 | self._model = self._model_fn(_env.model_dir) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 95, in wrapper f3k8ih5gq0-algo-1-9che5 | six.reraise(error_class, error_class(e), sys.exc_info()[2]) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/six.py", line 718, in reraise f3k8ih5gq0-algo-1-9che5 | raise value.with_traceback(tb) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper f3k8ih5gq0-algo-1-9che5 | return fn(*args, kwargs) f3k8ih5gq0-algo-1-9che5 | File "/opt/ml/code/inference.py", line 26, in model_fn f3k8ih5gq0-algo-1-9che5 | booster = pkl.load(open(os.path.join(model_dir, model_file), "rb")) f3k8ih5gq0-algo-1-9che5 | sagemaker_containers._errors.ClientError: [Errno 2] No such file or directory: '/opt/ml/model/xgboost-model' f3k8ih5gq0-algo-1-9che5 | 172.18.0.1 - - [23/May/2022:09:31:41 +0000] "GET /ping HTTP/1.1" 500 141 "-" "python-urllib3/1.26.9" f3k8ih5gq0-algo-1-9che5 | [2022-05-23:09:31:46:INFO] No GPUs detected (normal if no gpus installed) f3k8ih5gq0-algo-1-9che5 | [2022-05-23:09:31:46:INFO] Installing module with the following command: f3k8ih5gq0-algo-1-9che5 | /miniconda3/bin/python3 -m pip install . f3k8ih5gq0-algo-1-9che5 | Processing /opt/ml/code f3k8ih5gq0-algo-1-9che5 | Preparing metadata (setup.py) ... done f3k8ih5gq0-algo-1-9che5 | Building wheels for collected packages: inference f3k8ih5gq0-algo-1-9che5 | Building wheel for inference (setup.py) ... done f3k8ih5gq0-algo-1-9che5 | Created wheel for inference: filename=inference-1.0.0-py2.py3-none-any.whl size=13189 sha256=0343ebd456f9251526f64d5c6ae851b88410b3b053b71c21e41a94726ad2b5a2 f3k8ih5gq0-algo-1-9che5 | Stored in directory: /home/model-server/tmp/pip-ephem-wheel-cache-bs58b9ud/wheels/3e/0f/51/2f1df833dd0412c1bc2f5ee56baac195b5be563353d111dca6 f3k8ih5gq0-algo-1-9che5 | Successfully built inference f3k8ih5gq0-algo-1-9che5 | Installing collected packages: inference f3k8ih5gq0-algo-1-9che5 | Attempting uninstall: inference f3k8ih5gq0-algo-1-9che5 | Found existing installation: inference 1.0.0 f3k8ih5gq0-algo-1-9che5 | Can't uninstall 'inference'. No files were found to uninstall. f3k8ih5gq0-algo-1-9che5 | Successfully installed inference-1.0.0 f3k8ih5gq0-algo-1-9che5 | WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv f3k8ih5gq0-algo-1-9che5 | WARNING: You are using pip version 22.0.4; however, version 22.1.1 is available. f3k8ih5gq0-algo-1-9che5 | You should consider upgrading via the '/miniconda3/bin/python3 -m pip install --upgrade pip' command. f3k8ih5gq0-algo-1-9che5 | [2022-05-23 09:31:47 +0000] [225] [ERROR] Error handling request /ping f3k8ih5gq0-algo-1-9che5 | Traceback (most recent call last): f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper f3k8ih5gq0-algo-1-9che5 | return fn(*args, *kwargs) f3k8ih5gq0-algo-1-9che5 | File "/opt/ml/code/inference.py", line 26, in model_fn f3k8ih5gq0-algo-1-9che5 | booster = pkl.load(open(os.path.join(model_dir, model_file), "rb")) f3k8ih5gq0-algo-1-9che5 | FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/model/xgboost-model' f3k8ih5gq0-algo-1-9che5 | f3k8ih5gq0-algo-1-9che5 | During handling of the above exception, another exception occurred: f3k8ih5gq0-algo-1-9che5 | f3k8ih5gq0-algo-1-9che5 | Traceback (most recent call last): f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 56, in handle f3k8ih5gq0-algo-1-9che5 | self.handle_request(listener_name, req, client, addr) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/gunicorn/workers/ggevent.py", line 160, in handle_request f3k8ih5gq0-algo-1-9che5 | addr) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 107, in handle_request f3k8ih5gq0-algo-1-9che5 | respiter = self.wsgi(environ, resp.start_response) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_xgboost_container/serving.py", line 136, in main f3k8ih5gq0-algo-1-9che5 | user_module_transformer.initialize() f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_transformer.py", line 185, in initialize f3k8ih5gq0-algo-1-9che5 | self._model = self._model_fn(_env.model_dir) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 95, in wrapper f3k8ih5gq0-algo-1-9che5 | six.reraise(error_class, error_class(e), sys.exc_info()[2]) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/six.py", line 718, in reraise f3k8ih5gq0-algo-1-9che5 | raise value.with_traceback(tb) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper f3k8ih5gq0-algo-1-9che5 | return fn(args, kwargs) f3k8ih5gq0-algo-1-9che5 | File "/opt/ml/code/inference.py", line 26, in model_fn f3k8ih5gq0-algo-1-9che5 | booster = pkl.load(open(os.path.join(model_dir, model_file), "rb")) f3k8ih5gq0-algo-1-9che5 | sagemaker_containers._errors.ClientError: [Errno 2] No such file or directory: '/opt/ml/model/xgboost-model' f3k8ih5gq0-algo-1-9che5 | 172.18.0.1 - - [23/May/2022:09:31:47 +0000] "GET /ping HTTP/1.1" 500 141 "-" "python-urllib3/1.26.9" f3k8ih5gq0-algo-1-9che5 | [2022-05-23:09:31:52:INFO] No GPUs detected (normal if no gpus installed) f3k8ih5gq0-algo-1-9che5 | [2022-05-23:09:31:52:INFO] Installing module with the following command: f3k8ih5gq0-algo-1-9che5 | /miniconda3/bin/python3 -m pip install . f3k8ih5gq0-algo-1-9che5 | Processing /opt/ml/code f3k8ih5gq0-algo-1-9che5 | Preparing metadata (setup.py) ... done f3k8ih5gq0-algo-1-9che5 | Building wheels for collected packages: inference f3k8ih5gq0-algo-1-9che5 | Building wheel for inference (setup.py) ... done f3k8ih5gq0-algo-1-9che5 | Created wheel for inference: filename=inference-1.0.0-py2.py3-none-any.whl size=16359 sha256=56f1e8fab2fec8c4a15850ac4ecf9a02b73f0068434821b09cf73cbcda092824 f3k8ih5gq0-algo-1-9che5 | Stored in directory: /home/model-server/tmp/pip-ephem-wheel-cache-02pq641a/wheels/3e/0f/51/2f1df833dd0412c1bc2f5ee56baac195b5be563353d111dca6 f3k8ih5gq0-algo-1-9che5 | Successfully built inference f3k8ih5gq0-algo-1-9che5 | Installing collected packages: inference f3k8ih5gq0-algo-1-9che5 | Attempting uninstall: inference f3k8ih5gq0-algo-1-9che5 | Found existing installation: inference 1.0.0 f3k8ih5gq0-algo-1-9che5 | Can't uninstall 'inference'. No files were found to uninstall. f3k8ih5gq0-algo-1-9che5 | Successfully installed inference-1.0.0 f3k8ih5gq0-algo-1-9che5 | WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv f3k8ih5gq0-algo-1-9che5 | WARNING: You are using pip version 22.0.4; however, version 22.1.1 is available. f3k8ih5gq0-algo-1-9che5 | You should consider upgrading via the '/miniconda3/bin/python3 -m pip install --upgrade pip' command. f3k8ih5gq0-algo-1-9che5 | [2022-05-23 09:31:53 +0000] [227] [ERROR] Error handling request /ping f3k8ih5gq0-algo-1-9che5 | Traceback (most recent call last): f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper f3k8ih5gq0-algo-1-9che5 | return fn(*args, kwargs) f3k8ih5gq0-algo-1-9che5 | File "/opt/ml/code/inference.py", line 26, in model_fn f3k8ih5gq0-algo-1-9che5 | booster = pkl.load(open(os.path.join(model_dir, model_file), "rb")) f3k8ih5gq0-algo-1-9che5 | FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/model/xgboost-model' f3k8ih5gq0-algo-1-9che5 | f3k8ih5gq0-algo-1-9che5 | During handling of the above exception, another exception occurred: f3k8ih5gq0-algo-1-9che5 | f3k8ih5gq0-algo-1-9che5 | Traceback (most recent call last): f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 56, in handle f3k8ih5gq0-algo-1-9che5 | self.handle_request(listener_name, req, client, addr) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/gunicorn/workers/ggevent.py", line 160, in handle_request f3k8ih5gq0-algo-1-9che5 | addr) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/gunicorn/workers/base_async.py", line 107, in handle_request f3k8ih5gq0-algo-1-9che5 | respiter = self.wsgi(environ, resp.start_response) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_xgboost_container/serving.py", line 136, in main f3k8ih5gq0-algo-1-9che5 | user_module_transformer.initialize() f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_transformer.py", line 185, in initialize f3k8ih5gq0-algo-1-9che5 | self._model = self._model_fn(_env.model_dir) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 95, in wrapper f3k8ih5gq0-algo-1-9che5 | six.reraise(error_class, error_class(e), sys.exc_info()[2]) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/six.py", line 718, in reraise f3k8ih5gq0-algo-1-9che5 | raise value.with_traceback(tb) f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper f3k8ih5gq0-algo-1-9che5 | return fn(*args, *kwargs) f3k8ih5gq0-algo-1-9che5 | File "/opt/ml/code/inference.py", line 26, in model_fn f3k8ih5gq0-algo-1-9che5 | booster = pkl.load(open(os.path.join(model_dir, model_file), "rb")) f3k8ih5gq0-algo-1-9che5 | sagemaker_containers._errors.ClientError: [Errno 2] No such file or directory: '/opt/ml/model/xgboost-model' f3k8ih5gq0-algo-1-9che5 | 172.18.0.1 - - [23/May/2022:09:31:53 +0000] "GET /ping HTTP/1.1" 500 141 "-" "python-urllib3/1.26.9" f3k8ih5gq0-algo-1-9che5 | [2022-05-23:09:31:58:INFO] No GPUs detected (normal if no gpus installed) f3k8ih5gq0-algo-1-9che5 | [2022-05-23:09:31:58:INFO] Installing module with the following command: f3k8ih5gq0-algo-1-9che5 | /miniconda3/bin/python3 -m pip install . f3k8ih5gq0-algo-1-9che5 | Processing /opt/ml/code f3k8ih5gq0-algo-1-9che5 | Preparing metadata (setup.py) ... done f3k8ih5gq0-algo-1-9che5 | Building wheels for collected packages: inference f3k8ih5gq0-algo-1-9che5 | Building wheel for inference (setup.py) ... done f3k8ih5gq0-algo-1-9che5 | Created wheel for inference: filename=inference-1.0.0-py2.py3-none-any.whl size=19706 sha256=4e5bdcd2e59629fadf0c0e54a3d71f769dd92d167d08729e7ab0f4b9074a7066 f3k8ih5gq0-algo-1-9che5 | Stored in directory: /home/model-server/tmp/pip-ephem-wheel-cache-vq93idru/wheels/3e/0f/51/2f1df833dd0412c1bc2f5ee56baac195b5be563353d111dca6 f3k8ih5gq0-algo-1-9che5 | Successfully built inference f3k8ih5gq0-algo-1-9che5 | Installing collected packages: inference f3k8ih5gq0-algo-1-9che5 | Attempting uninstall: inference f3k8ih5gq0-algo-1-9che5 | Found existing installation: inference 1.0.0 f3k8ih5gq0-algo-1-9che5 | Can't uninstall 'inference'. No files were found to uninstall. f3k8ih5gq0-algo-1-9che5 | Successfully installed inference-1.0.0 f3k8ih5gq0-algo-1-9che5 | WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv f3k8ih5gq0-algo-1-9che5 | WARNING: You are using pip version 22.0.4; however, version 22.1.1 is available. f3k8ih5gq0-algo-1-9che5 | You should consider upgrading via the '/miniconda3/bin/python3 -m pip install --upgrade pip' command. f3k8ih5gq0-algo-1-9che5 | [2022-05-23 09:31:59 +0000] [225] [ERROR] Error handling request /ping f3k8ih5gq0-algo-1-9che5 | Traceback (most recent call last): f3k8ih5gq0-algo-1-9che5 | File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_functions.py", line 93, in wrapper f3k8ih5gq0-algo-1-9che5 | return fn(args, kwargs) f3k8ih5gq0-algo-1-9che5 | File "/opt/ml/code/inference.py", line 26, in model_fn f3k8ih5gq0-algo-1-9che5 | booster = pkl.load(open(os.path.join(model_dir, model_file), "rb")) f3k8ih5gq0-algo-1-9che5 | FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/model/xgboost-model'

Am I doing something wrong?

— Reply to this email directly, view it on GitHub https://github.com/aws-samples/amazon-sagemaker-local-mode/issues/15, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEICFBV3VI3AVT2N7EBWHLLVLNIBPANCNFSM5WVHQFDA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[pablocastilla]

Hi!

Thanks for answering and helping.

inside /opt/ml there are three folders: code, model and models. The last 2 are empty. How can I put the model in the tar.gz there?

I have also seen this: 172.18.0.1 - - [24/May/2022:14:29:12 +0000] "GET /ping HTTP/1.1" 500 141 "-" "python-urllib3/1.26.9"

[pablocastilla]

Actually I see the model created in S3. I suppose it should be created in the docker.

[eitansela]

When the endpoint is deployed then model.tar.gz file is download from s3 to the container by SageMaker and untar to /opt/ml/model directory.

Try to download the model.tar.gz from s3, untar it and see what happens.

On Tue, 24 May 2022 at 17:16 Pablo Castilla @.***> wrote:

Actually I see the model created in S3. I suppose it should be created in the docker.

— Reply to this email directly, view it on GitHub https://github.com/aws-samples/amazon-sagemaker-local-mode/issues/15#issuecomment-1136061945, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEICFBUBKUSJNS3H24SLOKDVLTXFFANCNFSM5WVHQFDA . You are receiving this because you commented.Message ID: @.***>

[pablocastilla]

If I do it that way it works and I can see one prediction. Looking at the code I suppose I should see two of them, but we are going forward.