Dataset updates which fail will be stuck in pending mode

[kardan]

Context

If an update is not successful it might get stuck in pending mode forever. An example for this is when a user is updating from Flow and tries to use a token with invalid grants. Then it's possible to issue an update, but the Flow API will not allow the update to happen.

Problem or idea

The problem is in how we're mixing JDBC transactions and Clojure Exceptions:

1. The job starts at https://github.com/akvo/akvo-lumen/blob/develop/backend/src/akvo/lumen/update.clj#L137
2. We branch to a future at https://github.com/akvo/akvo-lumen/blob/develop/backend/src/akvo/lumen/update.clj#L139
3. We open a JDBC transaction at https://github.com/akvo/akvo-lumen/blob/develop/backend/src/akvo/lumen/update.clj#L140
4. We start a try catch block at https://github.com/akvo/akvo-lumen/blob/develop/backend/src/akvo/lumen/update.clj#L96
5. If there is an exception we catch and set execution job to FAILED in the database at https://github.com/akvo/akvo-lumen/blob/develop/backend/src/akvo/lumen/update.clj#L129
6. We throw the exception again https://github.com/akvo/akvo-lumen/blob/develop/backend/src/akvo/lumen/update.clj#L130
7. We reach up at the JDBC transaction with an exception, hence everything including our attempt to set the job execution to 'FAILED' is rolled back.

Solution or next step

• We should make sure to mark the job as failed
• We should make sure this exception is handled and sent Sentry properly.

[kardan]

If the user reloads the page the spinner will be gone and things will look ok, but nothing will have been updated.