Visualization of data interpolation options

[colleenskemp]

In the context of this project, data interpolation is primarily used to take data that is issued at infrequent time points (say every decade), and interpolate it to a series with estimates for every year in the range we care about. We have the functions that can do this interpolation in the python library (model/interpolation.py, model/s-curve.py), already.

The Excel workbooks also have a tab which allows researchers to visualize how interpolations would look with different possible interpolation options, in order to select the parameterization that best fits the data. This ticket is to add some similar capability to the python code, most likely expressed as a Jupyter notebook.

Find a video containing overview of data interpolator here: https://drive.google.com/drive/folders/1widnlsMHoAQylb--FbmgDQ_uq0oIMeQm?usp=sharing

[colleenskemp]

This is a high priority tool.

[DentonGentry]

https://github.com/ProjectDrawdown/solutions/blob/develop/model/interpolation.py, I think.

[colleenskemp]

We may choose to include population harmonization use case in hackathon. @denised to follow up with further direction and separate out the 2 key elements of this ticket.

[denised]

@DentonGentry is correct, we have the base interpolation functions. What we do not have is an interface that allows users to see and select from different trend lines. It would be an interesting experiment to see if we can do a light-weight version of this in Jupyter Notebook. I will update the ticket description accordingly.

[eparamasari]

Daniel and I are taking this issue.

[danielmk]

I built this app that runs in a jupyter notebook (see screenshot) which allows to navigate through all scenarios, variables, sources and regions. On click of 'Plot', the four different interpolations are shown. No coding required on the researcher side. Is this what was planned for this issue? Are there any crucial features missing? @colleenskemp mentioned population harmonization. I would need a quick explainer on that to include it in the app. Also, could I get access to the dropbox with the interpolator video? That might give me a better idea of how it should look.

[Sunishchal]

@danielmk @eparamasari Here is the link to the explainer videos which can be publicly accessed: https://drive.google.com/drive/folders/1widnlsMHoAQylb--FbmgDQ_uq0oIMeQm?usp=sharing

[danielmk]

Thanks @Sunishchal that gives me a good idea. Regarding the population harmonization in Python, the only population data I found is this: https://github.com/ProjectDrawdown/solutions/blob/develop/data/unitadoption_ref_population.csv I guess the first step would be to create .csv data source for the other population scenarios featured in the video?

[denised]

@danielmk There are two population files there (unitadoption_ref_population.csv and unitadoption_pds_population.csv) but you are correct that there are more population scenarios in the new tool. I think it would be a good idea to create a new population subdirectory in data/ to contain the different variants. I'll send you Chad's Excel tool in slack, which has the data.

We would then need to update the code to be able to use those different variants, which is a slightly different task than visualizing them. If you are up for that, let me know and I'll make a new ticket for that and get you set up.

[danielmk]

I'm still very interested in this topic so if you open another issue you can assign me.

[denised]

Daniel, that is fantastic to hear! I have been revamping the issues list, and you should be able to see a number of new issues there — there are several different areas, so please see what interests you! Also, I have just started to use the wiki on the repo to give a more holistic view of what’s going on with the code right now—please check that out.

I would be very glad to have your help!

-- Denise Draper @.***

On Tue, Aug 3, 2021, at 10:45 AM, Daniel Müller-Komorowska wrote:

I'm still very interested in this topic so if you open another issue you can assign me.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ProjectDrawdown/solutions/issues/332#issuecomment-892038671, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKAMTTGETQGA5524LNE7F3T3ATKJANCNFSM5AV6WW3A.