Although setup is done by the script, the secrets management story is not good: one has to copy the secrets and manually create the relevant files directly in the machine.
Better would be to have those secrets stored on the vault where they'll be protected and, as part of a deployment pipeline, they would be installed in the target host. This entails:
CI pipeline setup: stages' configuration, secrets, etc.
Request a SSH key to the machine, for the pipeline, from DevOps
Have a way of logging into the machine and starting a new benchbot process with the pipeline's secrets as environment variables.
Although setup is done by the script, the secrets management story is not good: one has to copy the secrets and manually create the relevant files directly in the machine.
Better would be to have those secrets stored on the vault where they'll be protected and, as part of a deployment pipeline, they would be installed in the target host. This entails:
Suggestion: reopen https://github.com/paritytech/devops/issues/945 after this is done.