feat: support codeartifact for installing requirements.txt packages

humanzz commented 12 months ago

Issue #, if available: #85

Description of changes:

add a dependency on boto3 for calling codeartifact apis
update modelserver to check for the presence of codeartifact (CA* prefixed) environment variable
if env variable is present, build the authenticated endpoint index url, and add that to the pip install command

Testing done:

unit tests
container testing in https://github.com/aws/sagemaker-inference-toolkit/pull/130#issuecomment-1636911075

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

[x] I have read the CONTRIBUTING doc
[x] I used the commit message format described in CONTRIBUTING
[ ] I have used the regional endpoint when creating S3 and/or STS clients (if appropriate)
[ ] I have updated any necessary documentation, including READMEs

Tests

[x] I have added tests that prove my fix is effective or that my feature works (if appropriate)
[ ] I have checked that my tests are not configured for a specific region or account (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

humanzz commented 12 months ago

I've been using the codeartifact-related code for a few months already. I was not able to add any related integration tests as such tests would require access to a fixed codeartifact repo, and for the container to have access to AWS credentials.

I'm asking for guidance if any such integ tests would be needed here.

Additionally, is anything required in README to mention this CodeArtifact support?

humanzz commented 11 months ago

I did some testing by modifying test/container/dummy/Dockerfile to

+COPY dummy/requirements.txt /opt/ml/models/code/requirements.txt
+
 ENTRYPOINT ["python", "/usr/local/bin/dockerd-entrypoint.py"]
 CMD ["serve"]

where requirements.txt is a copy of https://github.com/aws/sagemaker-training-toolkit/blob/master/test/container/dummy/requirements.txt

I then went ahead and started the container - once without setting the CA_* env variables and once with

Without (i.e. public pypi index) docker run -p 8080:8080 sagemaker-inference-toolkit-test:dummy serve

Collecting pyfiglet==0.8.post1 (from -r /opt/ml/models/code/requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/33/07/fcfdd7a2872f5b348953de35acce1544dab0c1e8368dca54279b1cde5c15/pyfiglet-0.8.post1-py2.py3-none-any.whl (865kB)
Installing collected packages: pyfiglet
Successfully installed pyfiglet-0.8.post1
You are using pip version 19.0.3, however version 21.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Warning: MMS is using non-default JVM parameters: -XX:-UseContainerSupport
2023-07-15T22:48:58,867 [INFO ] main com.amazonaws.ml.mms.ModelServer -
...

With (i.e. from CodeArtifact index) docker run -p 8080:8080 -e "CA_REPOSITORY_ARN=xxx" sagemaker-inference-toolkit-test:dummy serve

/usr/local/lib/python3.6/site-packages/boto3/compat.py:88: PythonDeprecationWarning: Boto3 will no longer support Python 3.6 starting May 30, 2022. To continue receiving service updates, bug fixes, and security updates please upgrade to Python 3.7 or later. More information can be found here: https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/
  warnings.warn(warning, PythonDeprecationWarning)
Looking in indexes: https://***:****@xxx-xxx.d.codeartifact.xxx.amazonaws.com/pypi/xxx/simple/
Collecting pyfiglet==0.8.post1 (from -r /opt/ml/models/code/requirements.txt (line 1))
  Downloading https://xxx-xxx.d.codeartifact.xxx.amazonaws.com/pypi/xxx/simple/pyfiglet/0.8.post1/pyfiglet-0.8.post1-py2.py3-none-any.whl (865kB)
Installing collected packages: pyfiglet
Successfully installed pyfiglet-0.8.post1
You are using pip version 19.0.3, however version 21.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Warning: MMS is using non-default JVM parameters: -XX:-UseContainerSupport
2023-07-15T23:01:39,867 [INFO ] main com.amazonaws.ml.mms.ModelServer -
...

chen3933 commented 11 months ago

I think it makes more sense to add a new feature which allow user to run plugin module at model server start time than add more specialized function here.