aws / sagemaker-pytorch-inference-toolkit

Toolkit for allowing inference and serving with PyTorch on SageMaker. Dockerfiles used for building SageMaker Pytorch Containers are at https://github.com/aws/deep-learning-containers.
Apache License 2.0
134 stars 72 forks source link

fix: log4j migration from 1 to 2. Replace properties file with xml. #114

Closed nikhil-sk closed 2 years ago

nikhil-sk commented 2 years ago

Issue #, if available:

Description of changes:

  1. TS has upgraded from using log4j1.2 to log4j2, as part of 0.5.1 release.
  2. The pytorch inference toolkit supplies the properties file to be used for logging config, to torchserve, and needs to be upgraded.

LOGS

Confirmed by building a DLC that consumes this package (specifically a package built out of this PR):

it has the following expected output on loading a model:


Sagemaker TS environment variables have been set and will be used for single model endpoint.
--
Collecting transformers==4.6.0  Downloading transformers-4.6.0-py3-none-any.whl (2.3 MB)
Requirement already satisfied: requests in /opt/conda/lib/python3.8/site-packages (from transformers==4.6.0->-r /opt/ml/model/code/requirements.txt (line 1)) (2.27.1)
Collecting regex!=2019.12.17  Downloading regex-2021.11.10-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (764 kB)
Collecting tokenizers<0.11,>=0.10.1  Downloading tokenizers-0.10.3-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (3.3 MB)
Collecting huggingface-hub==0.0.8  Downloading huggingface_hub-0.0.8-py3-none-any.whl (34 kB)
Collecting filelock  Downloading filelock-3.4.2-py3-none-any.whl (9.9 kB)
Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.8/site-packages (from transformers==4.6.0->-r /opt/ml/model/code/requirements.txt (line 1)) (4.62.3)
Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.8/site-packages (from transformers==4.6.0->-r /opt/ml/model/code/requirements.txt (line 1)) (1.20.3)
Requirement already satisfied: packaging in /opt/conda/lib/python3.8/site-packages (from transformers==4.6.0->-r /opt/ml/model/code/requirements.txt (line 1)) (21.3)
Collecting sacremoses  Downloading sacremoses-0.0.47-py2.py3-none-any.whl (895 kB)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.8/site-packages (from packaging->transformers==4.6.0->-r /opt/ml/model/code/requirements.txt (line 1)) (3.0.6)
Requirement already satisfied: charset-normalizer~=2.0.0 in /opt/conda/lib/python3.8/site-packages (from requests->transformers==4.6.0->-r /opt/ml/model/code/requirements.txt (line 1)) (2.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.8/site-packages (from requests->transformers==4.6.0->-r /opt/ml/model/code/requirements.txt (line 1)) (2021.10.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/conda/lib/python3.8/site-packages (from requests->transformers==4.6.0->-r /opt/ml/model/code/requirements.txt (line 1)) (1.26.7)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.8/site-packages (from requests->transformers==4.6.0->-r /opt/ml/model/code/requirements.txt (line 1)) (3.3)
Collecting click  Downloading click-8.0.3-py3-none-any.whl (97 kB)
Requirement already satisfied: joblib in /opt/conda/lib/python3.8/site-packages (from sacremoses->transformers==4.6.0->-r /opt/ml/model/code/requirements.txt (line 1)) (1.1.0)
Requirement already satisfied: six in /opt/conda/lib/python3.8/site-packages (from sacremoses->transformers==4.6.0->-r /opt/ml/model/code/requirements.txt (line 1)) (1.16.0)
Installing collected packages: regex, filelock, click, tokenizers, sacremoses, huggingface-hub, transformers
Successfully installed click-8.0.3 filelock-3.4.2 huggingface-hub-0.0.8 regex-2021.11.10 sacremoses-0.0.47 tokenizers-0.10.3 transformers-4.6.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
2022-01-13T01:32:17,628 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager - Initializing plugins manager...
2022-01-13T01:32:17,683 [INFO ] main org.pytorch.serve.ModelServer -
Torchserve version: 0.5.2
TS Home: /opt/conda/lib/python3.8/site-packages
Current directory: /
Temp directory: /home/model-server/tmp
Number of GPUs: 0
Number of CPUs: 1
Max heap size: 29694 M
Python executable: /opt/conda/bin/python3.8
Config file: /etc/sagemaker-ts.properties
Inference address: http://0.0.0.0:8080
Management address: http://0.0.0.0:8080
Metrics address: http://127.0.0.1:8082
Model Store: /.sagemaker/ts/models
Initial Models: model.mar
Log dir: /logs
Metrics dir: /logs
Netty threads: 0
Netty client threads: 0
Default workers per model: 1
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Limit Maximum Image Pixels: true
Prefer direct buffer: false
Allowed Urls: [file://.*\|http(s)?://.*]
Custom python dependency for model allowed: false
Metrics report format: prometheus
Enable metrics API: true
Workflow Store: /.sagemaker/ts/models
Model config: {     "model": {         "1.0": {             "defaultVersion": true,             "marName": "model.mar",             "minWorkers": 1,             "maxWorkers": 1,             "batchSize": 3,             "maxBatchDelay": 100000,             "responseTimeout": 60         }     } }
2022-01-13T01:32:17,688 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager -  Loading snapshot serializer plugin...
2022-01-13T01:32:17,691 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: model.mar
2022-01-13T01:32:23,662 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model model loaded.
2022-01-13T01:32:23,676 [INFO ] main org.pytorch.serve.ModelServer - Initialize Inference server with: EpollServerSocketChannel.
2022-01-13T01:32:23,744 [INFO ] main org.pytorch.serve.ModelServer - Inference API bind to: http://0.0.0.0:8080
2022-01-13T01:32:23,744 [INFO ] main org.pytorch.serve.ModelServer - Initialize Metrics server with: EpollServerSocketChannel.
 

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot commented 2 years ago

AWS CodeBuild CI Report

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository