search

aws / sagemaker-pytorch-inference-toolkit

Toolkit for allowing inference and serving with PyTorch on SageMaker. Dockerfiles used for building SageMaker Pytorch Containers are at https://github.com/aws/deep-learning-containers.
Apache License 2.0
131 stars 70 forks source link

pass model directory as input to torchserve #118

Closed mseth10 closed 2 years ago

mseth10 commented 2 years ago

Issue #, if available: https://github.com/aws/sagemaker-pytorch-inference-toolkit/issues/117

Description of changes: Removing the _adapt_to_ts_format function and its test. It's not needed as torchserve accepts model directory as input now.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mseth10 commented 2 years ago

Error logs while running https://github.com/aws/sagemaker-pytorch-inference-toolkit/blob/master/test/integration/local/test_serving.py:

[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,605 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - Listening on port: /home/model-server/tmp/.ts.sock.9068
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,605 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - [PID]49640
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,605 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - Torch worker started.
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,605 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - Python runtime: 3.8.10
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,605 [INFO ] W-9068-model_1.0 org.pytorch.serve.wlm.WorkerThread - Connecting to: /home/model-server/tmp/.ts.sock.9068
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,606 [INFO ] W-9068-model_1.0 org.pytorch.serve.wlm.WorkerThread - Flushing req. to backend at: 1648863129606
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,606 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - Connection accepted: /home/model-server/tmp/.ts.sock.9068.
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,606 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - model_name: model, batchSize: 1
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - Backend worker process died.
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - Traceback (most recent call last):
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.8/site-packages/ts/model_loader.py", line 83, in load
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -     module, function_name = self._load_handler_file(handler)
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.8/site-packages/ts/model_loader.py", line 123, in _load_handler_file
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -     module = importlib.import_module(module_name)
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.8/importlib/__init__.py", line 127, in import_module
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -     return _bootstrap._gcd_import(name[level:], package, level)
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - ModuleNotFoundError: No module named 'model'
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - 
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - During handling of the above exception, another exception occurred:
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] epollEventLoopGroup-5-71 org.pytorch.serve.wlm.WorkerThread - 9068 Worker disconnected. WORKER_STARTED
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - 
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - Traceback (most recent call last):
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.8/site-packages/ts/model_service_worker.py", line 189, in <module>
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -     worker.run_server()
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.8/site-packages/ts/model_service_worker.py", line 161, in run_server
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -     self.handle_connection(cl_socket)
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.8/site-packages/ts/model_service_worker.py", line 123, in handle_connection
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -     service, result, code = self.load_model(msg)
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.8/site-packages/ts/model_service_worker.py", line 95, in load_model
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -     service = model_loader.load(model_name, model_dir, handler, gpu,
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [WARN ] W-9068-model_1.0 org.pytorch.serve.wlm.BatchAggregator - Load model failed: model, error: Worker died.
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.8/site-packages/ts/model_loader.py", line 85, in load
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -     module = self._load_default_handler(handler)
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.8/site-packages/ts/model_loader.py", line 128, in _load_default_handler
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -     module = importlib.import_module(module_name, 'ts.torch_handler')
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "/opt/conda/lib/python3.8/importlib/__init__.py", line 127, in import_module
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -     return _bootstrap._gcd_import(name[level:], package, level)
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG -   File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,607 [INFO ] W-9068-model_1.0-stdout MODEL_LOG - ModuleNotFoundError: No module named 'ts.torch_handler./opt/ml/model'
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,608 [WARN ] W-9068-model_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9068-model_1.0-stderr
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,608 [WARN ] W-9068-model_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - terminateIOStreams() threadName=W-9068-model_1.0-stdout
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,608 [INFO ] W-9068-model_1.0 org.pytorch.serve.wlm.WorkerThread - Retry worker: 9068 in 1597 seconds.
[36malgo-1-s7ww9_1  |[0m 2022-04-02T01:32:09,672 [INFO ] W-9044-model_1.0-stderr org.pytorch.serve.wlm.WorkerLifeCycle - Stopped Scanner - W-9044-model_1.0-stderr
sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository