The features that we want to hit over our current setup:
Use of shared infrastructure (AWS)
Use of shared devops software
HTTPS for Noctua
Improved Java version for Noctua (for minerva updates)
These should mostly be "done" already from previous work, but require testing and more documentation.
Additional targets we'll need to hit:
(Optionally) export of models from AWS image to GitHub or S3.
Create a "development" and "production" template set for configurations
Background Details
With the help of @abessiari, we built up a set of DevOps tools and scripts to setup and service our major components, namely the GO API, the RDF endpoint, and AmiGO/GOlr. This development work wrapped up in January, with the Noctua devops partially tested. We want to make sure that this new infrastructure is fully documented to meet the following criteria and use cases
Is usable for developers who have minimal experience with DevOps
Introduces the broad concepts of our DevOps
Terraform and workspaces (via S3 buckets)
Our naming conventions
How we refresh and test our services for a data release
Services not updated, but new one out with fresh data and old one destroyed
New instance up at patterned URL; test; switch Cloudflare/DNS; old instance down
Given the context above, can be used by a developer to
Deploy to a fresh instance at a new random domain for testing/development of software
Deploy a fresh instance to switch over to on a data release
TODO: Deploy a fresh instance with security updates
(Note that we also should have functional Docker images for local development, but that may be outside the scope of these Terraform-related docs.)
This documentation, out of necessity, may have to touch on topics like the release pipeline, credential distribution, DNS/Cloudflare, and the cert server.
Noting for @mugitty and @pgaudet that I'm in the process of spinning this into its own project. @mugitty and myself can meet to add more granular tickets. We can do a little of this tomorrow at the project meeting.
Project link
https://github.com/orgs/geneontology/projects/161
Project description
Summary
Developers at all levels should be able to test code, put out new code releases, and update data in Noctua in AWS using our documentation.
The bulk of the work is likely to be in https://github.com/geneontology/noctua_app_stack; some may also be in https://github.com/geneontology/devops-documentation, or even https://github.com/geneontology/noctua.
The features that we want to hit over our current setup:
These should mostly be "done" already from previous work, but require testing and more documentation. Additional targets we'll need to hit:
Background Details
With the help of @abessiari, we built up a set of DevOps tools and scripts to setup and service our major components, namely the GO API, the RDF endpoint, and AmiGO/GOlr. This development work wrapped up in January, with the Noctua devops partially tested. We want to make sure that this new infrastructure is fully documented to meet the following criteria and use cases
(Note that we also should have functional Docker images for local development, but that may be outside the scope of these Terraform-related docs.)
This documentation, out of necessity, may have to touch on topics like the release pipeline, credential distribution, DNS/Cloudflare, and the cert server.
PI
Chris
Product owner (PO)
Seth?
Technical lead (TL)
Anushya
Other personnel (OP)
Marginal: Sierra, Dustin
Technical specs
N/A
Other comments
This project is being derived from the Noctua portion of a larger AWS-ification project: https://github.com/orgs/geneontology/projects/146 https://github.com/geneontology/project-management/issues/85