concurrencylabs / aws-cost-analysis

Tools that make it easier to analyze AWS Cost and Usage reports. Initial version provides support for Athena and QuickSight.
GNU General Public License v3.0
173 stars 43 forks source link

Problem with directory path #5

Closed lfventura closed 6 years ago

lfventura commented 6 years ago

My AWS reports are being put into s3://<bucket>/<myprefix>/aws-cost-and-usage/ but the function works only if I dont have my own prefix. I had to edit it and do the following change to work out for me at utils.py:

-    period = dirs[len(dirs)-3]
+    period = dirs[len(dirs)-2]

-    for d in dirs[0:len(dirs)-3]:
+    for d in dirs[0:len(dirs)-2]:

Maybe there is a better fix to solve both cases #

concurrencylabs commented 6 years ago

Thank you for bringing this up. We will review this issue and add a comment here before the end of this week.

concurrencylabs commented 6 years ago

AWS places Cost and Usage reports in S3 according to the following key structure:

<prefix>/<period>/<hash>/<reportname>.csv.gz

... where prefix is optional and period is something like '20171201-20180101'

This function receives an S3 Put event when a new CUR is placed in your S3 bucket by AWS.

It seems the function is doing the right thing in order to extract the period, by doing

period = dirs[len(dirs)-3]

For example:

<prefix>/20171201-20180101/<hash>/<reportname>.csv.gz

len(dirs) is 4 len(dirs)-3 is 1, which is the right index for the period component '20171201-20180101'

or

20171201-20180101/<hash>/<reportname>.csv.gz

len(dirs) is 3 len(dirs)-3 is 0, which also finds the period component '20171201-20180101'

Could you please include an example of the s3 key in the S3 Put event the function received and also the error message you experienced?

Thanks

concurrencylabs commented 6 years ago

Path-related issues are fixed by https://github.com/concurrencylabs/aws-cost-analysis/pull/7