Add global data

[cypressf]

Data to add to global view:

• [x] red list index
• [ ] biodiversity index
• [ ] overlap of biodiversity and key areas
• [ ] creditary reporting system
• [ ] impact of natural disasters
• [ ] marine trophic index
• [ ] ecological footprint
• [ ] Worldbank 2020 gdp per capita data (purchasing power parity), or Conference Board data
• [ ] EPPA data: GDP
• [ ] EPPA data: Population

[cypressf]

Kilian put the data in https://www.dropbox.com/sh/ruwt4sdyhl8i9vp/AAAjO7xA9wrQsuDm6qDWPe_Ra?dl=0

[cypressf]

They are in python pickle format.

• [x] convert from pickle to csv file

[cypressf]

They have different index columns. Some are ISO-3 alphabetical, some are just country name, etc. I'm going to standardize everything around ISO 3166-1 numeric in https://github.com/cypressf/mit-climate-data-viz/issues/274#issuecomment-1220972612

[cypressf]

Some ISO-3 country codes aren't in the pycountry library I'm using:

XKX
CSHH
SCG
SUUH
YUG

I'll need to manually map those country codes to their ISO 3166-1 numeric counterparts.

[cypressf]

Okay, for now I've cleaned the data, and next I need to add it to the database. I might do this manually for now.

[cypressf]

I can't find the descriptions of the global biodiversity data. @azzacasch do you remember if Kilian put that somewhere in dropbox / google drive / microsoft teams? @Kiliyo sorry to ping you too, but did you have a document with descriptions of the biodiversity metrics somewhere?

[cypressf]

There's additional global data we're considering:

• EPPA Data
  • Global Trade Analysis Project (GTEP), a number of countries that we aggregate into the 18 regions of EPPA
  • What would we want to show from GTEP. It's different from the output of EPPA.
  • We look to other data sources to calibrate to when we use GTEP: IMF, IEA, EDGAR
  • When we do projections, we may use aggregated regions rather than countries, so perhaps we could estimate the projections for all the countries within a certain region to be the same
  • This aggregate projection could be misleading because people could think that it's more accurate to their country than it actually would be, given the nature of the aggregate, so we could add a note the growth rates are based on aggregates, not on an individual country model. For example, there are certain countries in latin america have all the hydro power.
  • We will start with GDP

[cypressf]

Biogen data will also be added

[cypressf]

Sergey will get me a mapping from EPPA regions to ISO country codes so I can translate between the EPPA region data to country data

[cypressf]

We need to make sure that people understand when they hover over "nigeria" for example, that it is showing GDP for all of africa for eppa region data. make sure to add an explanation for what is going on.

[cypressf]

I can't find the descriptions of the global biodiversity data. @azzacasch do you remember if Kilian put that somewhere in dropbox / google drive / microsoft teams? @Kiliyo sorry to ping you too, but did you have a document with descriptions of the biodiversity metrics somewhere?

@azzacasch pinging again. do you know if there are descriptions of Kilian's biodiversity data somewhere? I have the datasets available to use but I don't know what to title them or how to describe them. they are:

• CRS
  • Recipient name (EN)
  • USD_Received
  • total_area
  • Valuepa
• BII
• DIS
• EF
• MTI
• PAKA
• RLI

[cypressf]

If I were to guess, I would presume these correspond to:

Acronym	Full Name
CRS	Creditary Reporting System
BII	Biodiversity Index
DIS	impact of natural disasers?
EF	Ecological Footprint
MTI	Marine Trophic Index
PAKA	overlap of biodiversity and key areas?
RLI	Red List Index

for all datasets, except CRS, it's clear which column is the value. For CRS, I'm unsure whether to use USD_Receievd, total_area, or Valuepa.

[cypressf]

I am almost to the point where I can put this data in the database. I just need to remove rows whose country ids are not in our database.

[cypressf]

@azzacasch

To upload

1. go to est.mit.edu/uploader
2. use the UI to upload the csv data files
   • if there are any upload errors, contact me
   • if you want to correct the units, description, or name, go to est.mit.edu/dataset-editor
   • if you want to correct the name, description, or link of the data source, go to est.mit.edu/data-source-editor
3. create data visualizations here est.mit.edu/editor/-1
4. when you are happy with them, publish them to a tab of your choice (probably biodiversity for most of the global data)
   • if something went wrong and you don't like the visualization, you can always unpublish it by visiting the tab it's in in the editor, probably biodiversity, clicking the map, and clicking unpublish