copelco
commented
12 years ago Alright, made some progress on this:
- W Franklin exists in 2009 but not 2010 data. That's why I was unable to reproduce the issue as I was loading the files manually and not through the management command.
- After digging around, I discovered that TLIDs can have more than one associated feature name. So the TLID for a segment of W. Franklin has two names: W Franklin (primary) and 54 (alternate). The OpenBlock block importer ignores this possibility and simply takes the last feature name that it finds. So the order matters and is the reason why we had the 400 block of W. Franklin but not the 100 block. This also means that the block actually existed, it was just under "100 block of 54".
I talked with Paul about this bug and plan to submit a pull request to help fix the issue.
I researched this issue today. Using W. Franklin as a reference, I dug into the block importer and logs to see if any errors were being thrown for it. Not only did I discover zero errors, but the 100 W. Franklin block existed in the database. I was unable to reproduce the data bug found on the staging server. My guess is that the import failed when we initially imported the data and we didn't notice.
In review, I do think it would be useful to have a searchable record of the import. This would allow us to quickly and easily review the data importing stage to see if a particular street/block had errors. I think we can either create a log handler to do this or extend the block importer.