search

300Geeks / aws-elk-billing

Tools for Loading and Visualising AWS Detailed Billing with ELK(Elasticsearch, Logstash, Kibana)
MIT License
157 stars 26 forks source link

needs to handle multi-file reports #16

Open cg2v opened 7 years ago

cg2v commented 7 years ago

My billing report for april ended up split into two files. reportKeys in the manifest has two values:

"reportKeys":["billing/report/20170401-20170501/f6deac7f-7f11-4dfe-a3d2-332fd6540cfd/report-1.csv.gz","billing/report/20170401-20170501/f6deac7f-7f11-4dfe-a3d2-332fd6540cfd/report-2.csv.gz"],

I'm not actually using this fork yet, but the code in get_latest_zip_filename looks the same (https://github.com/PriceBoardIn/aws-elk-billing/blob/master/tools/tools.py#L158)

droidlabour commented 7 years ago

What exactly you mean by multi-file reports?

cg2v commented 7 years ago

The report is split in 2 files, which are both in the manifest. e.g.:

% aws s3 ls s3://bucket/billing/report/20170401-20170501/f6deac7f-7f11-4dfe-a3d2-332fd6540cfd/ 2017-04-30 12:16:07 573942 report-1.csv.gz 2017-04-30 12:24:05 114164377 report-2.csv.gz 2017-04-30 12:24:14 7727 report-Manifest.json

droidlabour commented 7 years ago

@cg2v Thanks for pointing it out. This seems to be an issue with bulk Billing Reports. I'm marking it as a bug and will release a patch ASAP.

johnnybraga commented 7 years ago

Any progress on this issue? Tks, - JB