[WIP] Fix some orphan KVStore issues

[bennybp]

Description

(depends on PR #652)

Due to the way finished calculations are parsed, data is always added to the kv_store table. If the base result previously contained foreign keys to the kv_store table, they are not deleted. This PR fixes that.

For procedures, we do not use upsert. This is to maintain atomic-like updating of procedures. If we upserted, then there could be issues where the stdout,error were updated, but the rest of the procedure info was not due to error, unexepected shutdown, etc.

But we can use the new upsert when updating services. Previously, new 1-2 rows per running service were added to the kv_store table every service iteration. Now we upsert, overwriting the previous output. There should be no atomicity problems here since the main source of the output is stored in the service table, and the next iteration would make everything consistent again.

Typically, the duplicated rows in kv_store are text for calculation errors (1-2K). These are small, so storage requirements were not growing appreciably.

Changelog description

Delete unused kv_store objects and upsert kv_store for services

Status

• [ ] Code base linted
• [ ] Ready to go

[codecov[bot]]

Codecov Report

Merging #653 (571966a) into master (78e4b18) will increase coverage by 0.02%. The diff coverage is 90.28%.

[lgtm-com[bot]]

This pull request introduces 3 alerts when merging c48c2e9c6fb0fe170b40d6275c4a426349bd4302 into a8e0af89c4500dfa965ff93ee8dfc51850b15502 - view on LGTM.com

new alerts:

• 1 for Unused local variable
• 1 for Unused import
• 1 for Wrong number of arguments in a call