cloudfoundry/capi-workspace

CAPI Workstation Helpers 🐋

Hello there! This repo is intended to help with development work on CAPI projects.

capi

Directory Overview

Description of the folders in this project:

Folder	Description
assets	random static files
bash-it	all the bash-it stuff, only bash-it stuff
helpers	random helper files
lib	scripts that could be sourced into your shell
bin	scripts that can be added to your `$PATH`
install-scripts	executable scripts that install a thing or two

Why is lib separate from bash-it? Not everyone uses bash-it, so lib is similar to scripts, giving people a way to manually load the things they care about in.

Minimal Install

One may manually (and minimally) load in capi-workspace content by including the following in their ~/.zshrc (or equivalent):

source <(cat $HOME/workspace/capi-workspace/lib/*) > /dev/null
path+=("$HOME/workspace/capi-workspace/bin")

CAPI Helper Commands

This table is generated from assets/capidoc.yml. It is dynamically available in your terminal using the capi command.

backup-restore
    compare_db_rows              Compares counts of two db's rows to validate backup & restore
bosh-deploy
    create_and_deploy            Create a capi release, upload to current bosh target, and deploy with default ops files (interactive)
    create_and_upload            Create a capi release and upload to current bosh target
    deploy                       Create a capi release, upload to current bosh target, and deploy with default ops files (non interactive)
    deploy_only_new_capi         Deploy just the current CAPI to current bosh target
    quick_deploy                 Quickly upload changes in cloud_controller_ng to relevant vms and restart"
bosh-lite-pool
    attach_debugger              Attaches the rubymine debugger to your bosh lite
    bootstrap_cf                 Target a CF, creates an org, space, and targets them
    cf_admin_password            Get the CF admin password for a current bosh target (requires credhub)
    cfu                          Alias for seed_users
    claim_bosh_lite              Claim an available bosh lite CF
    mysql_bosh_lite              Connect to current bosh target's MySQL DB
    print_env_info               List all the claimed bosh-lites in the pool
    psql_bosh_lite               Connect to current bosh target's MySQL DB
    rebootstrap_cf               Delete the 'org' org then call bootstrap_cf. For cleaning the slate.
    seed_users                   Add users of every role to the current org and space
    target_bosh                  Target a bosh environment from the pool
    target_cf                    Change cf cli target to the cloud foundry deployed on currently targeted bosh (hint: use "target_bosh" first).
    target_uaa                   Change uaac target to currently targeted bosh and login as admin
    unclaim_bosh_lite            Return a bosh lite CF back to the pool
    which_bosh                   Print which bosh environment is currently targeted
capi-release-author
    sync_package_specs           Add go submodule dependencies to bosh package specs
    sync_submodule_config        Sync git submodules with go dependencies using gosub
    unused_blobs                 Detects unused blobs from the blobs.yml file
file-system
    v                            Open matching location in vim (fasd -e vim)
    z                            Change current directory to directory matching provided pattern (ex: z ng)
git
    fixcommitter                 Cleans up committer/author after a rebase
    g                            git status
    gd                           git diff
    gdc                          git diff --cached
    git-open                     Open a git repo in your browser, courtesy of Paul Irish.
    pullify                      Pull down all PRs for current git repo as branches
    staged_shortlog              Get commit shortlogs and authors from submodule changes
    update                       Update all the git submodules
    which_capi                   Given a cloud_controller_ng commit, print which version of capi-release it first appears in
ruby-dev
    b                            bundle exec
    bake                         bundle exec rake
    gi                           gem install
    mybake                       DB=mysql bundle exec rake
    pgbake                       DB=postgres bundle exec rake
testing
    baras                        Runs baras with local integration_config
    cats                         Runs cats with local integration_config
    cats_cleanup                 Deletes builpacks, orgs, quotes, and service brokers in a targeted CF
    check_certificate_expirations        Reads YAML files, checks for expired (or premature) certs. Useful for pipeline troubleshooting
    delete_orgs                  Delete all orgs for current cf target
    generate_integration_config          Create an integration_config.json for running CATS against current bosh target.
    run_bridge_tests             Run Bridge unit tests
    sits                         Runs sits against a bosh lite
workspaces
    capi-workspaces              List and connect to existing K8s CAPI workspaces
    create-capi-workspace        Create a CAPI workspace in K8s cluster

Using the Bosh Lite Pool

As a perk, CAPI approvers get access to a pool of cf-deployment environments, deployed on bosh lites in a CFF-managed IaaS account. These environments are disposable, and are only intended for development use.

Though it is technically possible to use the pool without them, there are a number of convenience scripts provided by capi-workspace that make the pool usable by humans.

Prerequisites

git
Clone capi-workspace (this repo)
Add capi-workspace scripts/functions to your shell (see installation instructions above (or below))
Clone https://github.com/cloudfoundry/capi-env-pool

Example Pool Dev Workflow

Claim an environment from the pool:

❯ claim_bosh_lite
Claiming 'distaffs'...
Writing out .envrc...
Pushing reservation to capi-env-pool...
Use 'target_bosh distaffs' to set bosh environment variables.

Observe that you successfully claimed the environment:

❯ print_env_info
Rounding up claimed environments...

* ENV *      * CLAIMED BY *  * CLAIMED ON *                  * CLAIMED SINCE *
apologia     Tim Downey      tdowney.hostname                3 weeks ago
distaffs     Greg Cobb       gcobb.hostname                  2 minutes ago
riskier      Greg Cobb       gcobb.hostname                  13 days ago
zoned        Seth Boyles     capi-ws-cc-203                  3 weeks ago

Set bosh environment variables in your shell to "target" that environment:

❯ target_bosh distaffs

Refreshing bosh lite pool state...
Already up to date.

Sourcing /.../capi-env-pool/bosh-lites/claimed/distaffs to set bosh environment variables...

Writing a capi-specific integration_config.json...
Generated /.../capi-env-pool/distaffs/integration_config.json. Enjoy!

Writing a bosh gateway ssh key...
Generated /.../capi-env-pool/distaffs/bosh.pem. Enjoy!

Setting BOSH_GW_PRIVATE_KEY, BOSH_ALL_PROXY, and CONFIG environment variables...
Success!

❯ target_cf
Getting CF admin password from credhub...
Setting API endpoint to https://api.distaffs.app-runtime-interfaces.ci.cloudfoundry.org...
OK

API endpoint:   https://api.distaffs.app-runtime-interfaces.ci.cloudfoundry.org
API version:    3.166.0

Not logged in. Use 'cf login' or 'cf login --sso' to log in.
API endpoint: https://api.distaffs.app-runtime-interfaces.ci.cloudfoundry.org

Authenticating...
OK

Use 'cf target' to view or set your target org and space.

Create a default Organization and Space:

❯ bootstrap_cf
Getting CF admin password from credhub...
Setting API endpoint to https://api.distaffs.app-runtime-interfaces.ci.cloudfoundry.org...
OK

API endpoint:   https://api.distaffs.app-runtime-interfaces.ci.cloudfoundry.org
API version:    3.166.0

Not logged in. Use 'cf login' or 'cf login --sso' to log in.
API endpoint: https://api.distaffs.app-runtime-interfaces.ci.cloudfoundry.org

Authenticating...
OK

Use 'cf target' to view or set your target org and space.
Creating org org as admin...
OK

TIP: Use 'cf target -o "org"' to target new org
API endpoint:   https://api.distaffs.app-runtime-interfaces.ci.cloudfoundry.org
API version:    3.166.0
user:           admin
org:            org
No space targeted, use 'cf target -s SPACE'
Creating space space in org org as admin...
OK

Assigning role SpaceManager to user admin in org org / space space as admin...
OK

Assigning role SpaceDeveloper to user admin in org org / space space as admin...
OK

TIP: Use 'cf target -o "org" -s "space"' to target new space
API endpoint:   https://api.distaffs.app-runtime-interfaces.ci.cloudfoundry.org
API version:    3.166.0
user:           admin
org:            org
space:          space

Deploy the cf-deployment and capi-release versions from your dev machine:

❯ deploy
Uploading new release to 35.185.252.196.

Syncing bosh blobs...
bosh sync-blobs
...

Creating bosh release...
create-release --force --name capi
...

Uploading release to bosh director...
bosh upload-release --rebase
...

Deploying uploaded release...
bosh deploy cf-deployment.yml -o...
...
Succeeded

Make local changes and deploy a new CAPI version:

❯ deploy_only_new_capi
Building a new CAPI from local filesystem and deploying to 35.185.252.196.
bosh deploy <(bosh manifest) -o ...use-created-capi.yml ...
...
Succeeded

Release the environment for deletion, once you are done with it:

❯ unclaim_bosh_lite riskier
Refreshing bosh lite pool state...
Hit enter to release 'riskier'
rm 'riskier/.envrc'
Pushing the release commit to capi-env-pool...
Done!

Automated Workstation Setup

If you want, there are also some opinionated scripts you can run to set up a new computer for CAPI development. These may be useful if you are dynamically provisioning remote workstations, or something like that.

Dependencies

MacOS
pip

Running the Installer

    mkdir -p ~/workspace && cd ~/workspace
    git clone git@github.com:cloudfoundry/capi-workspace.git && cd capi-workspace

You can now either install via ./install.sh or ./install-core.sh.

Or curl the bootstrap script, which will create the workspace directory and clone this repo for you. This is primarily intended for quickly setting up virtual workstations.

bash <(curl -s https://raw.githubusercontent.com/cloudfoundry/capi-workspace/main/bootstrap.sh)

Manual steps (to be automated later)

Open System Preferences / Users & Groups /
/ Login Items
- Add flycut from Applications
- Add spectacle from Applications
Log out/Log in
- This will cause Flycut and Spectacle to run and ask for permissions
Open up Rubymine manually and enter a License Server
Install the mine cli shortcut (RubyMine -> Tools -> Create Command-line Launcher...)
If you are using Goland do the same two previous steps for Goland
If you have access to the private CAPI backlog, add a secure note called tracker_api_token to LastPass with your Pivotal Tracker API token. This token will be used in scripts such as claim_bosh_lite to pull story titles currently in flight.

Contributing to this repo

kindly consider when a change is worth making to install-core.sh versus install.sh.
bash-it We use bash-it to organize and streamline our bash settings. This includes stuff like color schemes, aliases, shell settings, and the shell prompt formatting. Adding "plugins" to custom-bash-it-plugins will cause them to be installed in every new shell.

What this installation does

install-core.sh: intended to be more friendly for developers who don't want this repository to take over their machine. This is intended to be the minimal set of things required for working with a CAPI repository.

Only installs core utilities, languages, services, and binaries.
Is NOT responsible for installing & configuring any editors (nvim/vscode/intellij)
Is NOT responsible for installing tools for improving developer workflows (jq/rg)
Is NOT responsible for applying any git configuration
Is NOT responsible for loading in scripts that need to be source'd.
Is NOT responsible for modifying your $PATH to include script

./install.sh: runs install-core.sh and configures the machine with many more packages/preferences

install.sh or install-core.sh ?

install-core.sh is a subset of install.sh and is intended to be less invasive to the machine, without compromising on provisioning a workstation suitable to working on CAPI projects.

For fresh workstations && full-time CAPI developers, install.sh might make more sense.

For folks working on multiple projects, install-core.sh might make more sense.

For folks with existing workstations & configuration, install-core.sh might make more sense.