search

paylogic / pip-accel

pip-accel: Accelerator for pip, the Python package manager
https://pypi.python.org/pypi/pip-accel
MIT License
308 stars 35 forks source link

Added readonly flag for s3 cache #42

Closed lgtml closed 9 years ago

lgtml commented 9 years ago

The purpose of this is to allow CI to keep the s3 cache current and developers will be able to prime their cache from the s3 source.

coveralls commented 9 years ago

Coverage Status

Coverage decreased (-0.41%) when pulling aba8b7842e67be9f89cc22ce3c0a1d2570541000 on lgtml:s3_readonly into ffe1f19e5a1012b3a92eb2805be087859bdad9b3 on paylogic:master.

lgtml commented 9 years ago

Could not see any existing tests for the Cache s3 cache. Here are my manual tests.

Intial with PIP_ACCEL_S3_READONLY=1

2014-11-20 21:34:27 leigh-XPS13 pip_accel[11941] INFO Unpacking source distribution(s) ..
2014-11-20 21:34:27 leigh-XPS13 pip_accel[11941] INFO Executing command: pip install --no-install --no-index --find-links=file:///home/leigh/.pip-accel/sources --build-directory=/tmp/tmpN8Oqrg scriptine
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking scriptine
  Running setup.py egg_info for package scriptine

    warning: no previously-included files matching '*.pyc' found under directory 'test'
Successfully downloaded scriptine
2014-11-20 21:34:27 leigh-XPS13 pip_accel[11941] INFO Finished unpacking 1 source distribution in 0.21 seconds.
2014-11-20 21:34:27 leigh-XPS13 pip_accel[11941] INFO Installing from binary distributions ..
2014-11-20 21:34:28 leigh-XPS13 pip_accel.caches.s3[11941] INFO Checking if distribution archive is available in S3 bucket: Linux-x86_643/v7/scriptine:0.2.0:CPython-2.7.tar.gz
2014-11-20 21:34:28 leigh-XPS13 pip_accel.bdist[11941] INFO Building scriptine (0.2.0) binary distribution ..
2014-11-20 21:34:29 leigh-XPS13 pip_accel.bdist[11941] INFO Finished building scriptine (0.2.0) in 0.41 seconds.
2014-11-20 21:34:29 leigh-XPS13 pip_accel.caches.s3[11941] INFO Skipping S3 Upload, S3 Readonly enabled
2014-11-20 21:34:29 leigh-XPS13 pip_accel[11941] INFO Finished installing 1 requirement in 1.29 second.

Second Clean Run without PIP_ACCEL_S3_READONLY set

2014-11-20 21:34:43 leigh-XPS13 pip_accel[11974] INFO Unpacking source distribution(s) ..
2014-11-20 21:34:43 leigh-XPS13 pip_accel[11974] INFO Executing command: pip install --no-install --no-index --find-links=file:///home/leigh/.pip-accel/sources --build-directory=/tmp/tmpc66UES scriptine
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking scriptine
  Running setup.py egg_info for package scriptine

    warning: no previously-included files matching '*.pyc' found under directory 'test'
Successfully downloaded scriptine
2014-11-20 21:34:43 leigh-XPS13 pip_accel[11974] INFO Finished unpacking 1 source distribution in 0.19 seconds.
2014-11-20 21:34:43 leigh-XPS13 pip_accel[11974] INFO Installing from binary distributions ..
2014-11-20 21:34:43 leigh-XPS13 pip_accel.caches.s3[11974] INFO Checking if distribution archive is available in S3 bucket: Linux-x86_643/v7/scriptine:0.2.0:CPython-2.7.tar.gz
2014-11-20 21:34:44 leigh-XPS13 pip_accel.bdist[11974] INFO Building scriptine (0.2.0) binary distribution ..
2014-11-20 21:34:44 leigh-XPS13 pip_accel.bdist[11974] INFO Finished building scriptine (0.2.0) in 0.41 seconds.
2014-11-20 21:34:44 leigh-XPS13 pip_accel.caches.s3[11974] INFO Uploading distribution archive to S3 bucket: Linux-x86_643/v7/scriptine:0.2.0:CPython-2.7.tar.gz
2014-11-20 21:34:44 leigh-XPS13 pip_accel.caches.s3[11974] INFO Finished uploading distribution archive to S3 bucket in 0.3 seconds.
2014-11-20 21:34:44 leigh-XPS13 pip_accel[11974] INFO Finished installing 1 requirement in 1.68 second.

Third Clean run with PIP_ACCEL_S3_READONLY set and bucket warmed

2014-11-20 21:34:58 leigh-XPS13 pip_accel[12005] INFO Unpacking source distribution(s) ..
2014-11-20 21:34:58 leigh-XPS13 pip_accel[12005] INFO Executing command: pip install --no-install --no-index --find-links=file:///home/leigh/.pip-accel/sources --build-directory=/tmp/tmpgIpc12 scriptine
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking scriptine
  Running setup.py egg_info for package scriptine

    warning: no previously-included files matching '*.pyc' found under directory 'test'
Successfully downloaded scriptine
2014-11-20 21:34:58 leigh-XPS13 pip_accel[12005] INFO Finished unpacking 1 source distribution in 0.25 seconds.
2014-11-20 21:34:58 leigh-XPS13 pip_accel[12005] INFO Installing from binary distributions ..
2014-11-20 21:34:59 leigh-XPS13 pip_accel.caches.s3[12005] INFO Checking if distribution archive is available in S3 bucket: Linux-x86_643/v7/scriptine:0.2.0:CPython-2.7.tar.gz
2014-11-20 21:34:59 leigh-XPS13 pip_accel.caches.s3[12005] INFO Downloading distribution archive from S3 bucket ..
2014-11-20 21:35:00 leigh-XPS13 pip_accel[12005] INFO Finished installing 1 requirement in 1.37 second.

Forth clean run without PIP_ACCEL_S3_READONLY set and s3 cache warmed

Uninstalling scriptine:
  Successfully uninstalled scriptine
(venv)(s3_readonly)leigh@leigh-XPS13:~/git/pip-accel$ PIP_ACCEL_S3_BUCKET=pip-accel.immun.io PIP_ACCEL_S3_PREFIX="`uname -s`-`uname -m`3"pip-accel install scriptine
install: missing destination file operand after ‘scriptine’
Try 'install --help' for more information.
(venv)(s3_readonly)leigh@leigh-XPS13:~/git/pip-accel$ PIP_ACCEL_S3_BUCKET=pip-accel.immun.io PIP_ACCEL_S3_PREFIX="`uname -s`-`uname -m`3" pip-accel install scriptine
2014-11-20 21:35:18 leigh-XPS13 pip_accel[12045] INFO Unpacking source distribution(s) ..
2014-11-20 21:35:18 leigh-XPS13 pip_accel[12045] INFO Executing command: pip install --no-install --no-index --find-links=file:///home/leigh/.pip-accel/sources --build-directory=/tmp/tmpHQ4qv4 scriptine
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking scriptine
  Running setup.py egg_info for package scriptine

    warning: no previously-included files matching '*.pyc' found under directory 'test'
Successfully downloaded scriptine
2014-11-20 21:35:18 leigh-XPS13 pip_accel[12045] INFO Finished unpacking 1 source distribution in 0.19 seconds.
2014-11-20 21:35:18 leigh-XPS13 pip_accel[12045] INFO Installing from binary distributions ..
2014-11-20 21:35:19 leigh-XPS13 pip_accel.caches.s3[12045] INFO Checking if distribution archive is available in S3 bucket: Linux-x86_643/v7/scriptine:0.2.0:CPython-2.7.tar.gz
2014-11-20 21:35:19 leigh-XPS13 pip_accel.caches.s3[12045] INFO Downloading distribution archive from S3 bucket ..
2014-11-20 21:35:20 leigh-XPS13 pip_accel[12045] INFO Finished installing 1 requirement in 1.08 second.