RDPC: Backup and Manual Restore of all services

[akachru-github]

This ticket will represent the work required to backup and manually restore all RDPC services.

• [x] Kafka (filestore). Initial investigation completed here: https://github.com/icgc-argo/rdpc-infra/issues/71
• [x] Song (postgres)
• [x] Intermediate Song (postgres)
• [x] ETCD configuration
• [x] Jenkins configuration
• [x] Replication to Cumulus NFS instance for all backups

Unknowns, but we know work will be required:

• Workflow graph -- REDIS, RabbitMQ

[akachru-github]

Kafka

• Originally targeted Kafka connector for MDB. We may want to switch this to something more appropriate like another cold Kafka, we can fall back to this. Mirroring one topic called weblog.
• Mirroring events + metadata to target
• Kafka is append only

Song

• Rob backing this up to obj store already. ticket for conerting this to NFS

Replication to Cumulus NFS instance for all backups

• set this up with our current set of backups (ETCD, Jenkins)
• pull from the final place of rest (Pulling from Collab)

Unknowns, but we know work will be required:

• Workflow graph -- REDIS, RabidMQ
• QC module

[akachru-github]

QC Module was removed from the scope of the Molecular Data Release.