search

twisted-infra / braid

Automation scripts for twistedmatrix.com
Other
6 stars 12 forks source link

[#225] Document the vagrant staging VM. #242

Closed adiroiban closed 7 years ago

adiroiban commented 7 years ago

Scope

This started as a way to solve https://github.com/twisted-infra/braid/issues/225

But while working on it I found out that the vagrant setup is too complicated, so I tried to make it simple and document the process

Changes

ansible is removed from vagrant vm. I heard rumors that we are moving to docker, so I have removed ansible so that we have only fab+braid for deployment.

It uses the vagrant per VM private key to connect. It removes the port forwarding as I don't know why it is needed.

The TODO file was removed and moved to a ticket.

The README was split into CONTRIBUTING. The idea is that README talk about how to use it and general things (for example in production) while CONTRIBUTING is only about hacking it.

I have updated base to install python-virtualbox and cffi-dev as they are required by the modern services.

Still to do

Rather than having the local-staging.sh script, maybe is better to write a few words so that people could understand what is going on there.

You might want to start only the buildbot/buidslave part of the infrastructure and in that case the ./local-staging.sh does not help.

the current local-staging.sh looks to me like a local-web-staging.sh

How to test

Follow the CONTRIBUTING documentation and hope you can get a vagrant VM running.

For now, I have tested the buildmaster/builldslave

adiroiban commented 7 years ago

I am still working at vagrant VM and various braid scripts, but I think that this can have an initial round of reviews

adiroiban commented 7 years ago

I can add these changes to this PR if you'd like, or I can open a new one that includes them, or you can make whichever seem easiest and most useful.

This PR started in an attempt to document the dev process. The README was talking about precise. I didn't know that we are now on 16.04.

We can upgrade to 16.04 in this PR.

For this PR my main goal is to have the Vagrant documented and working and base.bootstrap working.

We can fix the virtualenv, buidlbot and other services in separate PR

Free free to push your changes

Thanks!

adiroiban commented 7 years ago
  1. Upgrade the base box to xenial64 (Ubuntu 16.04). This is what dornkirk is actually running.
  2. Install virtualenv from PyPI into each user's site packages directory, rather than relying on the system's virtualenv. virtualenv comes with setuptools, pip, and wheel, and installs these into each new environment it creates. That means the easiest way to get virtual environments with up-to-date setuptools etc. is to upgrade virtualenv, and the easiest way to do that is to install it into each user's site packages directory. Adding a fabric command that manages this will make it easier to stay up to date.

This can be done here. I was not sure what are the rules for pushing global changes, but I think that is safe to mess with the "service" users.

  1. Upgrade pypy to 5.9. This might help with memory leaks!

Better a separate PR... which will also document how the whole web part works :)

adiroiban commented 7 years ago

Thanks. Yes. Virtualenv needs to be done in a separate PR... and it can be done together with upgrading the distro.

will merge and create a separate PR for 16.04 and in that one we can also improve the base setup.