= VEuPathDB Dataset Installer Service (VDI) :source-highlighter: highlightjs :toc:
ifdef::env-github[] :tip-caption: :bulb: :note-caption: :information_source: :important-caption: :heavy_exclamation_mark: :caution-caption: :fire: :warning-caption: :warning: endif::[]
ifndef::env-github[] :icons: font endif::[]
== Tagging, Deployment, and Branch Rules
=== Branch Rules
==== qa
The qa
branch is to reflect what is currently on QA. Code may only be pushed
to this branch via pull request which depends on both approval and a successful
workflow build of the code.
==== production
The production
branch should always reflect what is on production. This
branch is to remain locked until we are ready to move code from QA to
production, at which point the following steps will be performed:
. The production
branch will be unlocked via the repository settings.
. The qa
branch will be merged into production
. The production
branch will be re-locked via the repository settings.
production
branch up to date with the
qa
branch. For deployment procedure steps see the <==== main
The main
branch is for active development. It is strongly advised that fixes
and features be developed on branches forked from main
and then merged back
into main
when complete to avoid conflict problems with other active
development.
=== Tagging
Release tags v*
are only to be made on the production
branch.
Other tags should be avoided.
=== Deployment
The following is a full list of steps necessary to perform a deployment of the QA VDI service code onto production.
. Unlock the production
branch via the repository settings. This setting can
be found in the "Branches" menu under the "production" branch rule. The
specific setting to uncheck is "Lock Branch".
. Merge the qa
branch into production
.
. Re-lock the production
branch via the setting described in step 1.
. Wait for the production
branch to build in Jenkins.
. Create a new release tag on the merge commit from qa
to production
.
. Wait for the new tag to build in Jenkins.
. Create a PR on the https://github.com/VEuPathDB/tagger[Tagger] repository,
forked from and targeting the prod
branch, updating the vdi-service
tag
to the new tag created in step 5.
. Assign the PR to systems and notify them of the PR.
== Links
=== Documentation
=== VDI Project Index
==== Services
==== Docker Images
==== Libraries
===== Service Libraries
===== Plugin Libraries
==== Plugins
===== Production
===== Examples
== Repo Structure
The VDI service repository is divided into 4 categories: components, daemons, lanes, and the core bootstrapper.
Components are small shared libraries that contain code specific to a purpose that is not the core focus of the VDI service.
Daemons are background processes that perform maintenance tasks such as dataset reconciliation or pruning old, soft-deleted datasets from S3.
Lanes are event handlers that are tailored to individual event types, performing some process or processes on a dataset specified in the subscribed events.
The bootstrapper is the core of the service, and the root level Gradle project. This piece is responsible for starting up and shutting down the various project modules.
=== Internal Libs
Shared library components used by one or more VDI daemons, lanes, the rest service, and/or the bootstrapper.
=== Daemons
Background tasks that run unsupervised.
=== Lanes
Dataset event handlers. Each lane is a separate process that subscribes to a Kafka channel and operates on datasets whose information is provided in the incoming events.
=== Rest Service
The rest service is the public API through which users and administrators communicate with and operate on the VDI system.
=== Bootstrapper
The bootstrapper is the core of the service, and the root level Gradle project. This piece is responsible for starting up and shutting down the various project modules.
== Development
=== Running the Stack
. make build
+
Builds the VDI service docker image.
. make up
+
Spins up the service.
. make down
+
Shuts down the service and removes all the vdi-specific containers, volumes,
and networks.