Use stable/influxdb chart

deis / monitor

Monitoring for Deis Workflow

https://deis.com

MIT License

22 stars 32 forks source link

Use stable/influxdb chart #180

Open jchauncey opened 7 years ago

jchauncey commented 7 years ago

The team over at influxdata has built a pretty extensible chart for influxdb and its in the stable repo. We should move to using this one instead of our custom chart.

Acceptance Criteria:

[x] Remove current influx chart from deis/monitor repo
[x] Add stable/influxdb chart to the requirements.yaml in deis/workflow pinned to 0.2.1
[ ] Validate chart is fetched when performing helm dep udpate in CI
[x] Work out how we will handle the off cluster check we currently have

jchauncey commented 7 years ago

For off cluster storage we can do the following:

  - name: influxdb
    version: 0.2.1
    condition: influxdb.on_cluster
    repository: https://kubernetes-charts.storage.googleapis.com

The condition there is checked before installing the chart. The default in values.yaml is set to true but when people wnat off cluster influx support they just set it to false.

jchauncey commented 7 years ago

@mboersma @slack

So I have run into 2 small problems we should address:

1) The pvc name is different in our current chart than what will be used in the upstream influx chart. This means that we would make a new volume claim and our users could potentially lose data with using this new chart.

1) The location of the database itself is different between our images. This means that even if we mount in the same volume claim the underlying directories wouldnt match up and it would see that there was no data within the influx database itself.

Fixing issue 1 can be done with a PR into the upstream influx chart to add a conditional around setting the name. However, fixing the second issue means we would probably want to keep using our influx image.

I'm not entirely sure how to proceed with this so was wondering what your thoughts might be.

slack commented 7 years ago

It may be worth exploring writing a small tool/script that could be delivered as manual upgrade steps or a pre-upgrade hook in the charts. This script could orchestrate the upgrade, which could look roughly like:

Create new PVC which matches influxdb upstream
Remove influxdb deployment
Launch a data migration pod that refs both PVCs and copies data
Removes old PVC (or leaves it for manual clean up)
Lets the new influxdb chart drop in and take over going forward

Thoughts?

jchauncey commented 7 years ago

Im not entirely sure I like that idea because it definitely injects something in to the upgrade process that makes it feel really strange. It's not like what happened when we went from helmc -> helm.

slack commented 7 years ago

Long-term, we want to be able to lean on upstream components. That should reduce our maintenance burden. Otherwise, we'll continue to maintain a fork of just one component in the stack. To get there with the best user experience (no data loss) it seems worthwhile to tackle the data migration problem once.

Could you expand on "feels really strange", what aspects are specifically strange:

adding additional steps/jobs to upgrade (manual or otherwise)
manipulating data as part of the upgrade process
breaking an unstated customer expectation