Yelp / nerve-tools

Tools for configuring SmartStack's Nerve based on local sources
Apache License 2.0
3 stars 12 forks source link

Ensure internal yelp builds use internal pypi #76

Closed mattmb closed 4 years ago

mattmb commented 4 years ago

Mostly for security reasons we don't want to build from public pypi internally. We do want to use it for travis of course. Had to upgrade docker compose to make it pass a build arg. I think I caught everything pip does, largely copying approach by vkhromov in https://github.com/Yelp/paasta/pull/2872

vkhromov commented 4 years ago

Please get rid of --extra-index-url in src/debian/rules

vkhromov commented 4 years ago

I'd also recommend to run tests on a devbox after executing

$ sudo docker ps --format '{{.ID}}' | xargs -r sudo docker kill
$ sudo docker ps -a --format '{{.ID}}' | xargs -r sudo docker rm
$ sudo docker images --format '{{.ID}}' | xargs -r sudo docker rmi

, redirect the runs outputs to fluffy, and post URLs here just to make sure that there are no cached results from pypi.python.org in some intermediate images.

mattmb commented 4 years ago

Thanks for the pointers @vkhromov (and the paasta PR that made it easier to work out how to do this!)

Yeah, I was running itests on my devbox, I think they are now not using external pypi, here's the fpbs for trusty/xenial: https://fluffy.yelpcorp.com/i/j0dSFBsmRsFZKbxLjLk26T51gmhnPR83.html https://fluffy.yelpcorp.com/i/Jgkg9LT7lK8cgz8pP3vqRCHFLcw47SLs.html

I think bionic is still broken locally so I'll take another look at that tomorrow

vkhromov commented 4 years ago

https://fluffy.yelpcorp.com/i/j0dSFBsmRsFZKbxLjLk26T51gmhnPR83.html

Step 12/26 : RUN easy_install zope.interface
---> Running in ead78bb32258
Searching for zope.interface
Reading https://pypi.python.org/simple/zope.interface/
Best match: zope.interface 5.1.0
mattmb commented 4 years ago

Cool, they're all working internally now too: https://fluffy.yelpcorp.com/i/pzM08XfbpZFkM2VKlDjQvvqkMSFTV698.html https://fluffy.yelpcorp.com/i/xZ7F8KN0nLn5RDv9G65P3GcZ3BPgk0hf.html https://fluffy.yelpcorp.com/i/Pwvbq3HrQcSX9CrTPgMlwqCX1QKhLKrc.html

No mention of python.org so hopefully we're good, I'm going to rebase and merge