After the Cloud Team fixed the permissions in the admin-jenkins-gene-expression service account (set at the top of the YAML file) the pod that scales up SolrCloud is now working. This can bring costs down significantly, because the SolrCloud cluster will be scaled to 0 replicas and it will only be scaled up before a Jenkins agent starts working. A cron job checks every hour if Jenkins agents are running, based on a regex pod name match, and if none are found, then the SolrCloud stateful set is scaled down. See https://github.com/ebi-gene-expression-group/atlas-k8s-ci-environment/pull/2/files.
I also tested to run the tests with a SolrCloud cluster with “premium” (i.e. fast SSD) storage, but it hardly makes any difference, if at all. However, SSD storage was set for ZooKeeper from the start because it’s encouraged in ZK documentation. We also heard from bad experiences in PRIDE when they used NFS with ZooKeeper, and it couldn’t keep up.
After the Cloud Team fixed the permissions in the
admin-jenkins-gene-expressionservice account (set at the top of the YAML file) the pod that scales up SolrCloud is now working. This can bring costs down significantly, because the SolrCloud cluster will be scaled to 0 replicas and it will only be scaled up before a Jenkins agent starts working. A cron job checks every hour if Jenkins agents are running, based on a regex pod name match, and if none are found, then the SolrCloud stateful set is scaled down. See https://github.com/ebi-gene-expression-group/atlas-k8s-ci-environment/pull/2/files.I also tested to run the tests with a SolrCloud cluster with “premium” (i.e. fast SSD) storage, but it hardly makes any difference, if at all. However, SSD storage was set for ZooKeeper from the start because it’s encouraged in ZK documentation. We also heard from bad experiences in PRIDE when they used NFS with ZooKeeper, and it couldn’t keep up.