DRP: backup / restore docker volumes

[Jean-Baptiste-Lasselle]

This task is about implementing automated operations to backup restore all the docker volumes like https://github.com/decoder-leco/poc-data-visualization/issues/16

The question is very interesting : what are the best tools to implement such standar operations ? Is terraform good at that ? does it even make sense? Actually, when i think of terraform, ther eis one thing where terraform indeed is very interesting, the fact that terrafrom keeps the state of an infrastructure.

Now, say that i terraform my VM, with the jupyterlab inside:

• as i finished to first terraform it, well there are no R packages in the docker volume: and therefore in the terraform state, we do not have any information about the state of the installed packages
• yet, we could have one thing in the terrafrom state: the information that there is a backedup/restorable device there: the set of all docker volumes.
• Oh, so well , we would need there an abstraction in the magical cloud: a new kind of resource, made of my VM, with the whole pyterlab blabla, and a "restorable storage" or "resilient storage":
  • in the terraform, you just define a resource like this :

resource "resilient_docker_vm" "myjupyterlab" {
  # ---
  # and you just list the docker named volumes, they automatically are backedup with the configured frequency, to the condfigured minio s3
  drp {
    backup_frequency = "<here an expression to speicify the frequency of the execution of the backup>"

    docker_volumes = [
      "rlang-packages-data",
      "deno-packages-data",
      "python-packages-data",
      "conda-packages-data"
    ],

    # ---
    # and the cloud provider gives the option to backup your data OUTSIDE of their cloud: a real 123 backup, not 
    # just in their cloud with region or datacenter sites replication as we usually have...
    backup123 {
      # --- 
      # same settings as a minio client for s3 backup https://min.io/docs/minio/linux/reference/minio-mc/minio-client-settings.html
      # -> static creds: 
      #      export MC_HOST_<alias>=https://<Access Key>:<Secret Key>@<YOUR-S3-ENDPOINT>
      # -> STS creds: 
      #      export MC_HOST_<alias>=https://<Access Key>:<Secret Key>:<Session Token>@<YOUR-S3-ENDPOINT>
      minio_access_key = "<access key id secret value fetched from vault>"
      minio_secret_key = "<access secret key secret value fetched from vault>"
      minio_service_fqdn = "fqdn of your minio service"
      s3_bucket_name = "<bucket_name>"
    }
  }

}