2i2c-org / infrastructure

Infrastructure for configuring and deploying our community JupyterHubs.
https://infrastructure.2i2c.org
BSD 3-Clause "New" or "Revised" License
105 stars 64 forks source link

(Pending functional Cost Explorer API alternative) Cleanup Athena things #4668

Closed consideRatio closed 1 month ago

consideRatio commented 2 months ago

As an outcome of https://github.com/2i2c-org/infrastructure/issues/4648, we are switching towards implementing cost monitoring using the Cost Explorer API, a Python intermediary JSON server, and Grafana infinity datasource plugin instead.

If we arrive at a functional state of that, we should cleanup our alternative approach of using Athena that was partially implemented, as tracked via https://github.com/2i2c-org/infrastructure/issues/4546.

Definition of done

Gman0909 commented 1 month ago

Has there been any progress on unblocking this? Where is the alternative being worked on/prioritized?

consideRatio commented 1 month ago

The epic about cost allocation https://github.com/2i2c-org/infrastructure/issues/4453 is being worked with high prio, and this issue is just representing cleaning up an approach we moved away from along the way (very low prio).

I've run out of things that were planned and committed as dedicated issues in this sprint so far related to the EPIC, these are seen in this screenshot from the EPIC issue: image


Note that I ran into a conflict in this sprint to combine the following goals:

  1. To give the EPIC high prio in practice
  2. To follow sprint team practices of only committing to clearly scoped and well defined issues.

The issue is that its hard to make a chain of clearly scoped and well defined issues if they build on each others outcome. A workaround to this could have been to declare a followup issue that couldn't be clearly scoped and well defined saying that "Erik to do followup work, whatever that is, to help drive the cost allocation EPIC".

I've not been able to discuss this yet in a retro, where this discussion belongs I think. I'll continue it there internally with engineering.

I we run out of sprint committed issues, I'll proceed working on things related to the cost allocation epic.

Gman0909 commented 1 month ago

Thanks Eric, I appreciate the detailed update, and totally agree the conflict you raised is a great topic for a retrospective.

consideRatio commented 1 month ago

@Gman0909 I've tried mitigating this situation for the next sprint via https://github.com/2i2c-org/infrastructure/issues/4790.

Gman0909 commented 1 month ago

Thanks @consideRatio, I appreciate the heads up.