This is a grab-bag of three fixes for issues affecting the analysis or the admin views:
Issue #909
The OSM extract download for places in The United Kingdom of Great Britain and Northern Ireland wasn't working because the package we're using for country names had its short name as "United Kingdom" but the OSM extract URL has it as "great-britain". So this adds a special case.
Issue #910
The logic for where to get the population file was going wrong because it was using a standard name for non-US population files (population.zip) but then both caching it and checking for a cached file under that name. So the first file to get a PFB_POP_URL value downloaded the file and used it as intended, but all subsequent runs were actually ignoring that variable and preferring the population.zip file that had been saved to the S3 cache.
This changes the logic to always prefer PFB_POP_URL and to only save and use cached population files for US locations.
Issue #908
The country search in the Neighborhoods admin view started acting weird with the new release--searching on the full name of the country rather than the alpha-2 code, which is what's shown in the table and what it used to search on. It turned out that was an intentional change in the package we're using to provide our country field, so this just downgrades that package to the last version before the change.
Demo
The OSM and population files are hard to demonstrate visually, but here's the country search doing the right thing:
Notes
We could also change the lookup for the country search to "iexact" rather than "icontains", which hasn't been overridden by the package to search against the full name. But right now entering one letter gets you all countries that contain that letter, whereas if we made it exact, entering one letter would clear the list. The former seems nicer to me, and I don't think there's any real cost to holding the package on an earlier version, so it seemed like the way to go.
Testing Instructions
OSM fix: make a neighborhood in the UK (here's the boundary file for Milton Keynes: milton-keynes.zip and start an analysis for it. It should successfully download and import the OSM extract (you can cancel it after that. see issue #911)
Population file: as noted above, the logic around this is somewhat involved, and it would be a fair bit of work to try to test every case. It probably makes sense to kick off a non-US job with a PFB_POP_URL value set and confirm that it doesn't get uploaded to your storage bucket as population.zip, and that even if there is a file named population.zip in your storage bucket (i.e. at s3://pfb-USERNAME-storage-us-east-1/data/population.zip), when the PFB_POP_URL is set for a job it uses that and doesn't try to download the file from S3.
Overview
This is a grab-bag of three fixes for issues affecting the analysis or the admin views:
Issue #909 The OSM extract download for places in The United Kingdom of Great Britain and Northern Ireland wasn't working because the package we're using for country names had its short name as "United Kingdom" but the OSM extract URL has it as "great-britain". So this adds a special case.
Issue #910 The logic for where to get the population file was going wrong because it was using a standard name for non-US population files (
population.zip) but then both caching it and checking for a cached file under that name. So the first file to get aPFB_POP_URLvalue downloaded the file and used it as intended, but all subsequent runs were actually ignoring that variable and preferring thepopulation.zipfile that had been saved to the S3 cache. This changes the logic to always preferPFB_POP_URLand to only save and use cached population files for US locations.Issue #908 The country search in the Neighborhoods admin view started acting weird with the new release--searching on the full name of the country rather than the alpha-2 code, which is what's shown in the table and what it used to search on. It turned out that was an intentional change in the package we're using to provide our country field, so this just downgrades that package to the last version before the change.
Demo
The OSM and population files are hard to demonstrate visually, but here's the country search doing the right thing:
Notes
We could also change the lookup for the country search to "iexact" rather than "icontains", which hasn't been overridden by the package to search against the full name. But right now entering one letter gets you all countries that contain that letter, whereas if we made it exact, entering one letter would clear the list. The former seems nicer to me, and I don't think there's any real cost to holding the package on an earlier version, so it seemed like the way to go.
Testing Instructions
PFB_POP_URLvalue set and confirm that it doesn't get uploaded to your storage bucket aspopulation.zip, and that even if there is a file namedpopulation.zipin your storage bucket (i.e. ats3://pfb-USERNAME-storage-us-east-1/data/population.zip), when thePFB_POP_URLis set for a job it uses that and doesn't try to download the file from S3.Checklist