aws / sagemaker-experiments

Experiment tracking and metric logging for Amazon SageMaker notebooks and model training.
Apache License 2.0
125 stars 36 forks source link

Installation copies an init file into site-packages #165

Open hajapy opened 1 year ago

hajapy commented 1 year ago

Describe the bug When installing sagemaker-experiments via pip, an unexpected __init__.py file is produced in the site-packages directory.

To Reproduce Steps to reproduce the behavior:

  1. pip install sagemaker-experiments
  2. Look inside site-packages directory and notice a copy of this init file: https://github.com/aws/sagemaker-experiments/blob/v0.1.38/src/__init__.py

Expected behavior The site-packages directory should not be polluted with this __init__.py file.

To reproduce in an isolated container.

❯ cat Dockerfile
FROM python:3.10-buster

WORKDIR /root
ENV VENV /opt/venv
ENV LANG C.UTF-8
ENV LC_ALL C.UTF-8
ENV PYTHONPATH /root

RUN apt-get update && apt-get install -y build-essential

RUN ls /usr/local/lib/python3.10/site-packages/__init__.py && echo "found" || echo "not found"
RUN pip install sagemaker-experiments
RUN ls /usr/local/lib/python3.10/site-packages/__init__.py && echo "found" || echo "not found"

RUN cat /usr/local/lib/python3.10/site-packages/__init__.py

Log output:

❯ docker build --no-cache  --progress=plain .
#1 [internal] load build definition from Dockerfile
#1 sha256:8b90f70ea81e06c944fac247ce2389f0821ea7dfc73b5cbfa976a67f47aecbe3
#1 transferring dockerfile: 37B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:55884b2133828d601d1764d9971b42d2745ea19bb0f5561dfde153d90d9671e4
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/python:3.10-buster
#3 sha256:a84a21de82a108579c55b34707e57330c39bedb512ab207232478abd3f083aa6
#3 DONE 0.4s

#4 [1/7] FROM docker.io/library/python:3.10-buster@sha256:c8c1b51526d9652d9691dcd2ee44d5bec3e1beb9b9ba2ab6455503f1f6fc5cbc
#4 sha256:0c8ac355a716e10e84dc60582b5d5f3c9860cf6501bf553683e8ebf5f2a4e270
#4 DONE 0.0s

#5 [2/7] WORKDIR /root
#5 sha256:4087c248315663106a3fa119d2feaebf1dbb1b0bb07a205b36dec1c41c666192
#5 CACHED

#6 [3/7] RUN apt-get update && apt-get install -y build-essential
#6 sha256:557c3f943d0d687564866a4e1a2d55d25822c1633fab7434b4bf61c397e5911d
#6 0.284 Get:1 http://deb.debian.org/debian buster InRelease [122 kB]
#6 0.331 Get:2 http://deb.debian.org/debian-security buster/updates InRelease [73.5 kB]
#6 0.353 Get:3 http://deb.debian.org/debian buster-updates InRelease [56.6 kB]
#6 0.457 Get:4 http://deb.debian.org/debian buster/main amd64 Packages [7911 kB]
#6 0.690 Get:5 http://deb.debian.org/debian-security buster/updates/main amd64 Packages [338 kB]
#6 0.719 Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [8788 B]
#6 1.680 Fetched 8509 kB in 1s (5790 kB/s)
#6 1.680 Reading package lists...
#6 2.258 Reading package lists...
#6 2.806 Building dependency tree...
#6 2.921 Reading state information...
#6 3.036 The following NEW packages will be installed:
#6 3.036   build-essential
#6 3.091 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
#6 3.091 Need to get 7576 B of archives.
#6 3.091 After this operation, 20.5 kB of additional disk space will be used.
#6 3.091 Get:1 http://deb.debian.org/debian buster/main amd64 build-essential amd64 12.6 [7576 B]
#6 3.222 debconf: delaying package configuration, since apt-utils is not installed
#6 3.247 Fetched 7576 B in 0s (167 kB/s)
#6 3.265 Selecting previously unselected package build-essential.
(Reading database ... 24614 files and directories currently installed.)
#6 3.279 Preparing to unpack .../build-essential_12.6_amd64.deb ...
#6 3.281 Unpacking build-essential (12.6) ...
#6 3.310 Setting up build-essential (12.6) ...
#6 DONE 3.4s

#7 [4/7] RUN ls /usr/local/lib/python3.10/site-packages/__init__.py && echo "found" || echo "not found"
#7 sha256:a8eb7c93b52de81a4859b70017b146a99929339a6b7c227b8e3da18136bda75b
#7 0.358 ls: cannot access '/usr/local/lib/python3.10/site-packages/__init__.py': No such file or directory
#7 0.359 not found
#7 DONE 0.4s

#8 [5/7] RUN pip install sagemaker-experiments
#8 sha256:c0cd24134cfe1826ea1e4bd6c64c8dbd2ba65f2118c7699e8fa42ce8ee15bf11
#8 1.468 Collecting sagemaker-experiments
#8 1.608   Downloading sagemaker_experiments-0.1.35-py3-none-any.whl (42 kB)
#8 1.627      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.2/42.2 kB 2.3 MB/s eta 0:00:00
#8 2.023 Collecting boto3>=1.16.27
#8 2.053   Downloading boto3-1.24.44-py3-none-any.whl (132 kB)
#8 2.077      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 5.9 MB/s eta 0:00:00
#8 2.539 Collecting botocore<1.28.0,>=1.27.44
#8 2.560   Downloading botocore-1.27.44-py3-none-any.whl (9.0 MB)
#8 3.108      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.0/9.0 MB 16.5 MB/s eta 0:00:00
#8 3.179 Collecting jmespath<2.0.0,>=0.7.1
#8 3.200   Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)
#8 3.242 Collecting s3transfer<0.7.0,>=0.6.0
#8 3.266   Downloading s3transfer-0.6.0-py3-none-any.whl (79 kB)
#8 3.272      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.6/79.6 kB 21.9 MB/s eta 0:00:00
#8 3.336 Collecting python-dateutil<3.0.0,>=2.1
#8 3.357   Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
#8 3.375      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 kB 16.2 MB/s eta 0:00:00
#8 3.439 Collecting urllib3<1.27,>=1.25.4
#8 3.459   Downloading urllib3-1.26.11-py2.py3-none-any.whl (139 kB)
#8 3.472      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 139.9/139.9 kB 13.1 MB/s eta 0:00:00
#8 3.739 Collecting six>=1.5
#8 3.759   Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
#8 3.848 Installing collected packages: urllib3, six, jmespath, python-dateutil, botocore, s3transfer, boto3, sagemaker-experiments
#8 4.610 Successfully installed boto3-1.24.44 botocore-1.27.44 jmespath-1.0.1 python-dateutil-2.8.2 s3transfer-0.6.0 sagemaker-experiments-0.1.35 six-1.16.0 urllib3-1.26.11
#8 4.610 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#8 DONE 5.0s

#9 [6/7] RUN ls /usr/local/lib/python3.10/site-packages/__init__.py && echo "found" || echo "not found"
#9 sha256:0ab9ddd16c266573d2f1357c9e8ae41a16199c7f399aee2c86fdf18097abcbae
#9 0.353 /usr/local/lib/python3.10/site-packages/__init__.py
#9 0.353 found
#9 DONE 0.4s

#10 [7/7] RUN cat /usr/local/lib/python3.10/site-packages/__init__.py
#10 sha256:78e45cbd12f9426337921fd05705702d76d2d2bbbf5955df8327ca59e12243c9
#10 0.214 # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
#10 0.214 #
#10 0.214 # Licensed under the Apache License, Version 2.0 (the "License"). You
#10 0.214 # may not use this file except in compliance with the License. A copy of
#10 0.214 # the License is located at
#10 0.214 #
#10 0.214 #     http://aws.amazon.com/apache2.0/
#10 0.214 #
#10 0.214 # or in the "license" file accompanying this file. This file is
#10 0.214 # distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
#10 0.214 # ANY KIND, either express or implied. See the License for the specific
#10 0.214 # language governing permissions and limitations under the License.
#10 0.214 """Placeholder docstring."""
#10 DONE 0.2s

#11 exporting to image
#11 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#11 exporting layers
#11 exporting layers 0.5s done
#11 writing image sha256:975a8d71762f57588bee8c1a9a1b2d720217d5ee1915e573ead635f500f53016 done
#11 DONE 0.5s