bcgov / invasivesbc

Inventory and database repository for the creation, treatment, and report on BC Invasive plants and animals.
Apache License 2.0
11 stars 4 forks source link

Fix Database Locking Issue on Context Data Inserts #346

Closed popkinj closed 3 years ago

popkinj commented 3 years ago

Database Locking Issues While testing the context data loading logic, it was noted that numerious attributes were being missed. The operation happens on the same row in the database. This caused Postgres to throw locking errors.

At first this was thought to be an autocommit issue. However recent Postgres versions have this set on on by default.

Possible Solutions

  1. Convert SQL insertions from asynchronous to synchonous.
  2. Build a multi-insert statement and make only one query to the database.

I'm personlly in favour of the later as it would be more performant to run one transaction instead of many.

Tasks

popkinj commented 3 years ago

The BCGW layers are now being inserted in one long SQL statement. The local layer queries were not causing locking issues... So the logic was kept the same.