search

ucsc-cgp / cloud-billing-report

Generates a summary billing report for various UCSC-CGP cloud accounts.
5 stars 4 forks source link

Add personal usage data to reports #27

Open natanlao opened 4 years ago

natanlao commented 4 years ago

We will enhance the existing automated reporting tool to generate and distribute personal usage reports, via the owner tag e-mails, about what resources any person has outstanding on any given day. These will replace the existing global reports for everyone who doesn't need an overview of the whole lab's usage, and they will serve to remind people whan they've left instances running and how much they in particular are spending. (We probably don't want to send these when the only resources are SSH keys.) The subject line can include the amount of money spent that day (e.g. "AWS report: You spent $123.45 on October 3, 2020"). The report body should include some boilerplate guidance about the appropriate dollar value figures to use when thinking about trade-offs between the lab's money and one's own time.

natanlao commented 3 years ago

From Benedict: for the sake of transparency, personal reports should highlight personal usage, and summary of individual use should be available to everybody.

natanlao commented 3 years ago

From Benedict: personal reports should also account for S3 usage.

natanlao commented 3 years ago

This can likely be accomplished by grouping on the resourceTags/user:owner column, unless changed by #33

natanlao commented 3 years ago

I was unable to complete this ticket. I wanted to complete #33 first to make this easier.

Implementing this may require a deeper architectural change and may warrant removing Docker functionality. Broadly speaking, these are the approaches I had in mind in order from least work to most work:

  1. Remove Docker dependency and interface with sendmail via subprocess
  2. Keep Docker dependency, connect container to host sendmail
  3. Keep Docker dependency, dump generated emails to a folder on the host and send them from the host

There are more approaches, but I was thinking about taking approach 1 and using it as an excuse to more tightly couple the failed report retry functionality.

Richard-Hansen commented 3 years ago

Clarification: We want to send emails to the owners of a resource to remind them that they have the resource up and running. Thus, we can only send remind users if they tag their resources with their email address. (CloudCustodian provides regex filtering, so we can easily compile a list of resources that have emails as tags)