vastup

[PI-Victor]

Purpose

e.g.: oc cluster up

Add functionality to vastctl to bring up the platform with all components isolated in docker containers running locally.

This feature would help with

• end-to-end testing. (i believe it can be integrated with travis without a big deal, since it will have native support for linux)
• is an integral part of the user experience (people would be able to have vastness up and running locally in less than 2 minutes without any external deps - except for docker)
• developer experience - build and test end to end locally the changes you made in one or more components. (build images locally with the latest binary changes, instead of pulling them from the registry)

Motivation

I started to add all of the components to the existing docker-compose and realized there might be a better way that provides a better out-of-the-box experience see above ^^.

Proposed functionality

e.g.:

$ vastctl up
Starting backing service
Fetching docker.io/posgres:9.6.8-alpine...

Loading coordinator component...
Fetching quay.io/vastness.io/coordinator:latest

Loading vcs-webhook component...
Fetching quay.io/vastness.io/vcs-webhook:latest

Loading linguist component...
Fetching quay.io/vastness.io/linguist:latest

Starting coordinator...
Service available on port 8081
Logs available at container mounted path  /tmp/coordinator/logs/

Starting vcs-webhook...
Service available on port 8082
Logs available at container mounted path  /tmp/vcs-webhook/logs/

Starting linguist...
Service available on port 8083
Logs available at container mounted path /tmp/linguist/logs/

Vastness is now up 
Endpoint available at https://127.0.0.1:8081

Optionally: vastctl up --stateful /Users/local-user/coordinator/persistent_db_data_path would mount a local path of your choice where the persistent data is stored from the container. In this case it's postgres backing service, but can be any type of persistent data path separated based on component.

Driver and OS support

docker/machine

• supports MacOS via hyperkit/vpnkit/datakit components from moby
• supports Windows through Hyper-V driver from machine
• supports linux natively via docker socket
• optional support - we could include virtualbox for generic cross-platform support.

Aiding local development (Future)

You can pass container local build context to the docker daemon

vastctl up build-context --coordinator /Users/test-user/projects/go/src/github.com/vastness-io/coordinator
vastctl up build-context --vcs-webhook /Users/test-user/projects/go/src/github.com/vastness-io/vcs-webhook
vastctl up build-context --linguist /Users/test-user/projects/go/src/github.com/vastness-io/linguist

This will save a yaml file with a context definition containing the path to your component. You could then code, build the component and do a vastctl up and that will:

• take the binary you just compiled
• "slice" the docker image and search for the SHA where you add the component bin - see the Dockerfile

  IMAGE               CREATED              CREATED BY                                      SIZE                COMMENT
  5af004cca8b1        About a minute ago   /bin/sh -c #(nop)  ENTRYPOINT ["/coordinat...   0 B
  d60ed0bc94d0        About a minute ago   /bin/sh -c #(nop)  EXPOSE 8080/tcp              0 B
  92981d500e9d        About a minute ago   /bin/sh -c #(nop) ADD file:8e36eb2a8d51b82...   16.1 MB
  b19838de751a        About a minute ago   /bin/sh -c apk update && apk add ca-certif...   633 kB
  77144d8c6bdc        3 months ago         /bin/sh -c #(nop)  CMD ["/bin/sh"]              0 B
  <missing>           3 months ago         /bin/sh -c #(nop) ADD file:6edc55fb54ec9fc...   3.97 MB

• build on top of 92981d500e9d like

  FROM 92981d500e9d
  ADD /path/to/component/bin /component
  ENTRYPOINT ["/component"]

• spin up a fresh new platform instance that will have the latest changes you've made (across components even)

I know this sounds like "a LOT" but i've done this before in https://github.com/openshift/origin/pull/9564/files unfortunately that didn't make it in.

[jw-s]

@PI-Victor Awesome Idea, completely agree! After thinking it through a bit, we should seperate these into a seperate CLI as vastctl has the intention of interacting with a running ecosystem rather than provisioning. So this could be the start of a provisioner cli, which could deploy the ecosystem on any arch with the concept of "backends" or something?

[PI-Victor]

i think backends is a great idea, i didn't think of that. regarding vastctl i thought the same that it's a lot of stuff to add to it and doesn't exactly fit the purpose. I already started to stub something out and will show a PoC when i have it working. I've done it in a private repo because i know myself and i never finish stuff so i don't wanna disappoint.

[jw-s]

@PI-Victor It would be nice to have a repo here and see the progression, that way I can also work on it too?

[PI-Victor]

sure, no problem! if you're interested i will migrate it to the org immediately. L.E.: available at: https://github.com/vastness-io/vastup