feat: Create function which translates profiles defined for one zone set to another zone set

[merrielle]

Pull Request doc

Purpose

Addresses https://github.com/Breakthrough-Energy/PreREISE/issues/259 "Create function which translates profiles defined for one zone set to another zone set"

DEMO JUPYTER NOTEBOOK W/ DATA: https://www.dropbox.com/s/srbbkxirykug843/zone_translation.ipynb?dl=0

What the code is doing

translate_zone_set() has three key steps:

• Overlay map of prev_zones with new_zones
• Pivot table to make it a matrix of prev_zones x new_zones
• Scale the matrix such that each row sums to 1, then return it

Also included some plotting and results analysis functions. Not sure if these belong in PostREISE?

• plot_zone_map() can do both interactive maps and static PNGs
• filter_interesting_zones() takes the output of translate_zone_set() and filters it to only show when a single zone has been distributed to several new zones

Testing

TODO

Where to look

translate_zone_set() is the most important

Usage Example/Visuals

Example of plot_zone_map() using Liming's BA shapefiles:

Example of plot_zone_map() using Merrielle's BA shapefiles:

Example of filter_interesting_zones(), showing translation from Liming's BA shapefiles to Merrielle's BA shapefiles: NOTE: We're rounding the results to three decimal places to make them more readable, so not all rows add up to 1.

Time estimate

30 mins?

[BainanXia]

Also included some plotting and results analysis functions. Not sure if these belong in PostREISE?

I think put these plot functions closer to its main context is good. The reason we have most visualizations in PostREISE is because the package is designed for post analysis of simulation results rather than a place where we should put all visualizations in. Open to discussion.

Not sure whether we will use functionalities here in scenario analysis in the future though. But if that's the case, we will definitely need more than geographical definitions, thus more complicated plots in PostREISE will be implemented anyways.

[BainanXia]

A higher level question for the team: should we put support modules like this in gather or should we create another folder, say utility next to gather as we did in PowerSimData?

[merrielle]

Looks like using Pseudo-Mercator does actually affect the results -- mostly by a couple thousandths. Included a quick screenshot of partial new results on top and partial old results on the bottom.

[merrielle]

@BainanXia @danielolsen Aside from pytest failing (it passes locally so I'm investigating) I think this is ready for a final pass. I've addressed all the PR comments, added tests, and added a helper function make_zones_valid() that can be used to clean any imported gdf.

Regarding Bainan's comment about changing how we distribute overlaps:

Another approach to bring it down to 1 is looking at the overlaps among columns and distribute the overlap portion uniformly cross columns that causing duplications rather than all areas.

I've been investigating this more closely and there are many many edge cases that we may or may not want to handle. The rabbit hole goes deep. I'll send you a summary of some of my findings and address this in a future PR.

[danielolsen]

Are the tests failing because we add geopandas to the Pipfile but we haven't generated a new Pipfile.lock?

[merrielle]

Possibly! Updating it now.