ZTL QA issue: high qc_bbldiff.shp record count

[jackrosacker]

I haven't covered every avenue on our side yet, but flagging @fvankrieken early so we can look in parallel, or if you know of a change that could be responsible.

Issue

qc_bbldiff shapefile contains 6,000+ lots (typically contains ~200 lots). A quick examination shows that:

• the lots are scattered across the entire city
• the majority of the lots do not seem to be explained by any particular zoning change
• there is no visual spatial difference between last month's lots and this month's
• a calculation and comparison of lot centroid between this and last month's lots, for a lot that appears in the qc_bbldiff file (BBL 1014930013), shows a difference in coordinates, but at an absurdly high level of precision, i.e. 994916.3270805925 vs 994916.3270805478 

Next Steps

• I'm contacting TRD to see if they made any sweeping changes that aren't apparent yet
• I'm digging into the data to find any other apparent patterns
• I'll re-export and make sure I didn't introduce some error in how I re-projected the DTM
• Do you have any immediate ideas as to why this would be happening, based on the changes you made recently?

[fvankrieken]

I'll start comparing dtm between last months and this months builds

[fvankrieken]

This was caused by the way ztl does QA. QA tables are created in a persisted db where every ztl is archived at the end of each build. Last month, after ztl was published, a new build was run using the mid-cycle dtm that I had to reproject. Not sure if something went wrong ingesting that dtm, but that build (from nightly qa) was archived in the ztl qa db. When a new build (with a fixed dtm) was run this month, it wasn't compared to last months published ztl but rather this archived nightly_qa build that was never published.

I reran a job recreating last month's build (pinning source versions) to replace last month's archive, then reran this month's build with new data. qc_bbldiffs file looks as expected now