Closed cmoussa1 closed 6 months ago
Thanks for reviewing @ryanday36 - the sharness test added in t1011-job-archive-interface.t
is in fact checking job usage values for correctness. I was also able to confirm that wrapping up the updates into a single transaction is OK for calculating both user and bank usage values - subsequent updates made into a single transaction should be readable by each other. 👍 I'll set MWP here
Problem
When flux-accounting needs to do a job usage update for all of the associations in the
association_table
, it needs to make a number of updates to a number of tables in the DB. If there are a lot of associations, this can slow down the performance of a number of other, much simpler commands, such asflux account view-user
. This is because the job usage updates makes a separate transaction for each UPDATE statement performed, forcing other reads from the DB to wait until the updates are completed.This PR drops the separate
commit()
calls from the various helper functions that execute UPDATE statements injob_archive_interface.py
and instead wraps the entire set of database updates into a single SQLite transaction, which will only be committed to the DB after all of the updates are done. This should hopefully begin to relieve some of the headache mentioned in #451.