Noah-Huppert
commented
5 years ago Overview
This will take a bit of planning.
Up until this point we have not been storing any data exclusively in the database. The database has not been the source of truth. Instead the database has been acting as a cache. The source of truth has been the serverless-apps GitHub repo.
We cannot and should not store things like download counts in the serverless-apps repo.
Where should we store the download count?
- Prometheus: We have a Prometheus cluster already set up. Prometheus is well suited to collect this data. As download count is a time series metric. However this existing Prometheus cluster is used to monitor apps. We would probably want a separate cluster just for the purpose of download metrics
- Pros:
- Well suited for collecting time series data
- Client library already exists which handles publishing time series metrics
- Cons:
- Should setup a separate cluster just for this purpose
- Prometheus does not have any data integrity features (backups, sharding, replication)
- Mongo DB: We are already using Mongo DB to cache apps. We could store app statistics in this same database.
- Pros:
- Already using Mongo DB, little work to add another field to the app model
- Already running Mongo DB
- Cons:
- Not currently running Mongo DB in a reliable way, no backups, no sharding, no replication, we delete Mongo DB instances willy-nilly b/c we know we can recreate data from the GH registry repo
- Have to ensure no race conditions occur when incrementing the counter. Prometheus does this for us
Things we have to plan for
We have to operate either a Prometheus or Mongo DB instance reliably. This means implementing backups. If we choose Mongo DB this also means running Mongo DB in cluster high availability mode.
Todo
- [ ] Decide where we should store metrics
- [ ] Decide which metrics we should store
PratikMahajan
Track how many people have deployed each app through KScout