space-meridian / roadmap

High-level roadmap for Filecoin Station
https://starmap.site/roadmap/github.com/space-meridian/roadmap/issues/1
0 stars 0 forks source link

Setup infra for spark-observer #109

Closed bajtos closed 5 months ago

bajtos commented 5 months ago

As part of https://github.com/filecoin-station/roadmap/issues/97 and https://github.com/filecoin-station/roadmap/issues/106, we need to build a new service to listen for Meridian events and update spark-stats database. The current design, where spark-evaluate manages and updates the spark-stats database is no longer serving us well.

Going forward, we want to implement the following design:

### Tasks
- [ ] https://github.com/filecoin-station/spark-impact-evaluator/issues/12
- [x] Migrate `spark-stats` database to `spark-evaluate` database. Update `spark-evaluate` and `spark-stats` database connection string in Fly secrets to use the new database.
- [x] Implement a new monorepo structure in the spark-api repository.
- [x] Rework the spark-stats repository to use this new monorepo structure too.
- [x] Add a new component `spark-observer` to the `spark-stats` repository, setup Fly.io deployment
- [x] Add database migration scripts to spark-stats repository.
- [x] Modify `spark-stats` to read from two databases: `spark-evaluate` for existing data, `spark-stats` for the future data created by spark-observer
### Nice-to-have improvements
- [ ] Add npm scripts like `test:api` and `lint:publish` to spark-api
- [ ] Add npm scripts like `test:stats` and `lint:observer` to spark-stats
juliangruber commented 5 months ago

@bajtos I'm assigning you as DRI, so this issue has one. Feel free to change it if you disagree