KitWallace / treemap

3 stars 2 forks source link

Update database from BCC data #161

Open KitWallace opened 6 years ago

KitWallace commented 6 years ago

Update the database from the latest version of the BCC data. This once again contains the tree measurements but still no survey date . This makes merging difficult.

KitWallace commented 6 years ago

major work to develop the process and scripts to do this

http://kitwallace.co.uk/trees/convert/batch-update.xq

contains the documentation and scripts

KitWallace commented 6 years ago

As of 24 May the scripts have been used to do the update. This is a complex task and is not yet available for running other than by me.
The conversion script is now pretty good at delivering a clean set of tree nodes and is held in the staging directory. Various comparison and update scripts have been written but need tiding: New trees: trees which exist in the upload but not in the database - this is working but places the data in its own tree document - it should really disperse the trees into their relevant site document

Missing trees - BCC trees in the database not in the upload. These can be previewed and applied, but the quality of this data needs verification before it should be applied

Matching trees - I divided these into two situations ones where the tree has never been edited and where either sitecode, girth, or state have changed ones where there is prexisting edits

The first , after eyeballing the differences between the old and new, can applied. In the 2018-05-15 data there where 2051 trees updated The second needs manual intervention. Of 90 trees in this category, only 4 were better than the edited version , so we have better data form the remaining 86 cases

KitWallace commented 6 years ago

Deletions - there were 1300 potential deletions but when I deleted the old sponsorship data, these nearly all disappeared - bit odd?

KitWallace commented 4 years ago

Trees now loaded via WFS since pagination is supported.

Trees which are in the lasted data have been added. More work to do on deletions and changes - difficult