These changes get the INSPIRE pipeline running a basic form. These initial steps of the pipeline work fine (although time will tell how reliable they are):
updating company ownership data
downloading the latest INSPIRE data
backing up raw INSPIRE data to the Hetzner storage box
transforming the raw GML data and inserting it into the pending_inspire_polygons table
The end result is that the new ownership data is visible to all LX users, and the pending polygons can be viewed by LX super users.
The following stages of the pipeline are very unreliable so shouldn't be run, since they will almost definitely crash the Node app:
analyse each of the pending_inspire_polygons and find a match
decide which of the pending_inspire_polygons to accept
write accepted polygons into the main DB table so the changes are visible to all LX users
In order to skip these steps, add the stopBeforeTask=analyseInspire query param when running the pipeline using the /run-pipeline API
This must be deployed to production at the same time as https://github.com/DigitalCommons/land-explorer-back-end/pull/60 to ensure API compatibility
These changes get the INSPIRE pipeline running a basic form. These initial steps of the pipeline work fine (although time will tell how reliable they are):
The end result is that the new ownership data is visible to all LX users, and the pending polygons can be viewed by LX super users.
The following stages of the pipeline are very unreliable so shouldn't be run, since they will almost definitely crash the Node app:
stopBeforeTask=analyseInspirequery param when running the pipeline using the/run-pipelineAPI