A couple fixes for the recent changes to the analysis--one related to the new options, one to the Django upgrade:
The new RUN_IMPORT_JOBS environment variable for the analysis is a number (0 or 1), which works fine for running the analysis locally but, it turns out, is not OK when sending actual jobs to Batch. I think this has bitten us once before, and since the values always need to be strings, rather than adjust this one value, I just added a str() call to the function that packages the environment variables up for the boto request.
The stress network tiles for newly run analysis jobs had some black lines on them. I had seen them earlier in development, but I thought they were due to stale OSM files, and I thought after I had cleared my OSM cache they went away. But my test runs must not have worked how I thought, because they definitely showed up on staging. E.g.
This turned out to be because the LayerMapping utility, which we use to re-import the shapefiles produced by the analysis so that they can be displayed by the tiler, evidently shifted its treatment of nulls with the Django upgrade. When there are null values in the shapefile properties for integer fields, it used to import them as zeroes, but now it imports them as nulls. So that seems like a bug fix, and you can hardly blame them for making the change, but it turns out we were relying on those zeroes. Adjusting the tiler configuration to be able to handle both cases would be somewhat complicated, and especially right now when we haven't touched it in a while, but updating the data to replace nulls with zeroes right after the import is simple, so I did that. Here's an "after" version (the score and map are also slightly different because it turned out there were stale OSM extracts in the staging bucket, too, which I cleared out):
Testing Instructions
There's a staging batch worker running right now, so it should work to submit a job and watch it run.
Or you could take the existence of three successful jobs within the past 24 hours and the fact that the map looks OK on https://staging.pfb.azavea.com/#/places/f99374b7-abe3-44ef-bc6a-4feb22545e4d/ as sufficient evidence that these ultimately very small code changes are working as intended.
Overview
A couple fixes for the recent changes to the analysis--one related to the new options, one to the Django upgrade:
RUN_IMPORT_JOBSenvironment variable for the analysis is a number (0 or 1), which works fine for running the analysis locally but, it turns out, is not OK when sending actual jobs to Batch. I think this has bitten us once before, and since the values always need to be strings, rather than adjust this one value, I just added astr()call to the function that packages the environment variables up for the boto request.Testing Instructions
There's a staging batch worker running right now, so it should work to submit a job and watch it run. Or you could take the existence of three successful jobs within the past 24 hours and the fact that the map looks OK on https://staging.pfb.azavea.com/#/places/f99374b7-abe3-44ef-bc6a-4feb22545e4d/ as sufficient evidence that these ultimately very small code changes are working as intended.
Checklist