Solidify Shiva usage story

[erikrose]

Should this be a package? If so, add a setup.py. Figure out a story about how it'll be used in practice: at what point does the shiva package get fetched? Does it get installed in some venv somewhere?

[erikrose]

Goal: Shiva should be self-contained within a project. Its requirements shouldn't spill over into the project's requirements.txt.

[rhelmer]

Should Shiva be a submodule, or perhaps packaged and installed via pip itself, with the deployee only providing plugins/subclasses/etc?

[erikrose]

A thought experiment:

What if deployments looked like this?

shiva dxr/deployment  # Contains: requirements.txt, deploy.py
    # Makes a venv.
    # Peep or pip installs dxr/deployment/requirements.txt
    # Runs venv/python dxr/deployment/deploy.py

Ah, but then you somehow have to know which version of the project to check out before any of that happens, and that can be nontrivial. For example, we might want the last ancestor of master that passed in Jenkins. Making the code to do all that a prerequisite to using Shiva defeats Shiva's purpose. So Shiva can't be the entrypoint (unless it wants to delegate to some entrypoint in deploy.py, which is impractical because deploy.py might have arbitrary dependencies); a project-dwelling script must be the entrypoint.

To be continued…

[erikrose]

Actually, Shiva can (and should) be the entrypoint. We simply have to break the deployment process into 2 phases: bootstrap and deploy. Bootstrap determines what version of the code to check out and checks it out. Deploy builds and installs it. Witness:

shiva dxr/deployment  # Contains: requirements.txt, deploy.py
    # Makes a venv.
    # Peep- or pip-installs dxr/deployment/requirements.txt
    # Runs `venv/python dxr/deployment/deploy.py bootstrap`, which checks out the latest good version of the project and outputs /path/to/checkout.
    # Makes a new venv.
    # Peep- or pip-installs dxr/deployment/requirements.txt
    # (We can skip the previous 2 steps if the requirements are unchanged. A pip download cache should make none of this matter much.)
    # Runs `venv/python /path/to/checkout/.../deploy.py`, which builds the project as contained in the checkout and installs it

Improvements still to be made:

• Make the deployment dir a Python package so we can import stuff from deploy.py and so we have a natural place to put tests for the deployment if we like.