search

project-koku / masu

This is a READ ONLY repo. See https://github.com/project-koku/koku for current masu implementation
GNU Affero General Public License v3.0
5 stars 6 forks source link

AWS Report Processor Failing on Finalized Bill #357

Closed adberglund closed 5 years ago

adberglund commented 5 years ago

Describe the bug There is an UPDATE statement in the AWS report processor that is failing trying to change the same row multiple times. 

[2019-02-04 19:55:16,880: INFO/ForkPoolWorker-1] File <_io.TextIOWrapper name='/var/tmp/masu/...aws/cost-usage-bucket/...-koku-1.csv.gz' encoding='UTF-8'> opened for processing
[2019-02-04 19:55:21,405: ERROR/ForkPoolWorker-1] masu.processor.tasks.get_report_files[...]: ON CONFLICT DO UPDATE command cannot affect row a second time
HINT: Ensure that no rows proposed for insertion within the same command have duplicate constrained values.

Expected behavior The report processor should update only the appropriate single row.

adberglund commented 5 years ago

I believe everything is working properly here, but we require resource_id in our cost usage reports to check for true row uniqueness when updating the database. I did a 1 time load of our demo account so that January data shows there. I configured out cost usage report to include resource id in the future and opened the following issue https://github.com/project-koku/koku/issues/610