Pda targets

[fscottfoti]

These are the changes designed to begin to address 9.1 and 9.2 of task order 3 which we're calling the vision solver.

It should be noted that at this point, we are not successfully meeting the vision for all PDAs, but we do have the infrastructure necessary to see how our predictions compare to the pre-defined targets, both at an aggregate level (per pda) and at a disaggregate level (per parcel-development). In addition, the exercise has helped to fix some underlying data issues (e.g. interpretation of zoning) so the PDA results are much closer than they were when starting this exercise.

The simulations are currently waiting on data for the scheduled development events (the large projects that we know will happen and which might change the real estate market around those developments) and the no project zoning (which upzones a few of the pdas to give them capacity enough to meet pda targets, which they do not always have now).

Note that a number of changes are actually to improve workflows in model diagnosis - to be able to run simulations more easily and to be able to see and diagnose results more easily.

Next follows a description of the changes that have been made:

• Slack integration so that when simulations begin and end, messages are written to a specific channel in Slack
• Hand modifications to the current HLCM where coefficients are made more negative on average income in the neighborhood and on price for lower income quartiles. This keeps price from dropping in high-income neighborhoods as high-income neighborhoods and although this is a manual tweak, results pass the eyeball test significantly better than before. For instance, Pacific Heights does not decrease over the course of the simulation.
• Supply and demand has been turned on in all recent runs to make prices sensitive to the demand models and to places where supply is low. A new parcel filter has also been added in settings.yaml which removes parcels from consideration where there is only one unit and where the parcel is less than a half acre (i.e. single family lots). This could be fixed in a more principled way by fixing the zoning in the relevant locations.
• Price shifters have been added both by county and by pda in order to allow for calibration and "vision solving." For now these have mainly been adjusted by hand.
• Some basic capacity calculations can now be done which look for the zoned dwelling units and the underbuild on those units with various configurations of the calculations. This can also now be done in Javascript in the parcel_explorer if you want to modify those calculations iteratively until the logic seems most correct.

A few important scripts have been added.

• This one takes the parcel level results of the simulations, sums the net residential units, and compares the result to the pda targets, writing out the modeled result, the targeted result and the ratio between them. This ratio also provides the necessary number to do calibration to targets when we are ready to do that (i.e. when the zoning allows it).
• This one takes the output from the script above and uses the Folium library in Python to make a themed web map showing where PDA targets are over- or under-shot. (Or met.) The result is a static html file that can be stored or checked into github with no dependencies.
• And finally this one serves the data from the Bay Area hdf5 file using the Python library Flask to serve JSON to a web map. The client to that server is checked into master here. This has a hardcoded server location, but then the actual html page can be served from anywhere (it has no built-in data) so right now the page is actual kept on GitHub Pages and can be reached here. Right now I am having trouble keeping the webservice up and running (it keeps crashing) so the page doesn't always work. I will keep working to fix the webservice as it's a pretty basic thing really and should be reliable enough.

[fscottfoti]

This now also has scheduled development events and an update to orca. All new data is available on s3 and in the Makefile, and requirements.txt is updated with the appropriate hashes.

This isn't the final product of the simulation, but given that various people are working on the various parts, I think it's best to go ahead and merge to master as the latest and greatest.

@buckleytom before I merge can you try this out tomorrow and make sure it runs for you.

@smmaurer once we commit this, this will be the point to start from.

[tbuckl]

@fscottfoti checking it now

[tbuckl]

@fscottfoti on os x, i merged it on top of a local master and ran make, simulation.py, and estimation.py and it seems fine. can test on windows tomorrow am.

[smmaurer]

@fscottfoti sounds good, thanks