Add a global map

[cypressf]

• [x] decide on global vs just country level
• [x] find a good geojson or shape file
• [x] decide how the database can work with countries
• [x] add a tab for "usa," and "global"

[cypressf]

We are going to use all of asia for now. The data we have for russia is for all of russia right now, not for all of rusia only in asia, but kilian will split out the data for the asian portion of rusia.

[cypressf]

I'm looking to find some geojson files for africa and asia

[cypressf]

Going to use the data behind https://observablehq.com/@fil/map-of-africa or https://observablehq.com/@djr4/gdp-per-capita-ppp-africa-vs-serbia for the map of africa. Potentially could use the same data with a different filter and projection for the map of asia.

[cypressf]

whatever map file I use, I plan to use the iso-3 format for countries

[cypressf]

I'm going to use https://github.com/topojson/world-atlas for my map. I'll find a way to filter by region if we want to show only africa and asia. For now, I'm going to experiment with it and see if I can get some data displaying on a country-level globally.

[cypressf]

database modification

• [x] Add country table

Column	Type	Nullable
id	int2	NO
name	text	NO

• [x] Populate country table

primary key: id

• [x] Rename county_data table to usa_county_data
• [x] Rename county table to usa_county
• [x] Rename state table to usa_state
• [x] Add country_data table

Column	Type	Nullable	Foreign key
dataset	int4	NO	dataset(id)
source	int4	NO	data_source(id)
start_date	date	NO
end_date	date	NO
country_id	int2	NO	country(id)
value	float8	YES

• [x] Add geography_type table

Column	Type	Nullable
id	int4	NO
name	varchar(100)	NO

• [x] insert (1, "usa-county"), (2, "country")

• [x] Modify map_visualization table, add column

Column	Type	Nullable	Foreign key
geography_type	int4	NO	geography_type(id)

• [x] Assign geography type "county" (1) to all existing maps

• [x] Modify data_category table, add column

Column	Type	Nullable	Foreign key
geography_type	int4	NO	geography_type(id)

• [x] Assign geography type "county" (1) to all existing data categories

[cypressf]

controller modification

• [x] Add country_data_controller

#[get("/country-data/{id}")]
fn get(id,{source, start_date, end_date})

• [x] Rename data_controller to usa_county_data_controller

Similarly add and rename daos, and models.

Note: instead I'm putting everything inside the same data_controller.rs

[cypressf]

Frontend modification

• [ ] if geography_type is country, show world countries map and request data from /api/country-data, otherwise show the usa counties map and request data from /api/usa-county-data
• [ ] if world tab is selected, only show tabs that are part of the country geography type, if usa tab is selected, only show tabs that are part of the usa-county geography type.

[cypressf]

world-atlas uses ISO 3166-1 numeric 3-digit country codes instead of the iso-3 alphabetical format. Let's use those in the database and for all data as well. Perhaps I could add a column for alphabetical format if needed, but I'll keep them numeric for now.

[cypressf]

there are 249 countries in kilian's dataset

[cypressf]

There are 174 countries in the world map I'm using. It's worth examining which countries are missing between Kilian's datasets and my world map to see if we can show a more complete picture. Is this because the world map is outdated? Because it's missing labels? Because Kilian's dataset is outdated?

[cypressf]

Based on my test (https://github.com/cypressf/map-data-cleaning/commit/8aa2f6abf28535a2344e133add956a32a7a81b43), the following 75 countries are missing from the map:

🇵🇳 Pitcairn 🇸🇭 Saint Helena, Ascension and Tristan da Cunha 🇲🇶 Martinique 🇰🇲 Comoros 🇯🇪 Jersey 🇻🇬 Virgin Islands, British 🇲🇺 Mauritius 🇸🇯 Svalbard and Jan Mayen 🇨🇻 Cabo Verde 🇲🇴 Macao 🇨🇰 Cook Islands 🇧🇲 Bermuda 🇵🇼 Palau 🇻🇮 Virgin Islands, U.S. 🇹🇰 Tokelau 🇦🇸 American Samoa 🇲🇫 Saint Martin (French part) 🇬🇺 Guam 🇾🇹 Mayotte 🇨🇼 Curaçao 🇭🇰 Hong Kong 🇸🇲 San Marino 🇧🇧 Barbados 🇲🇨 Monaco 🇼🇫 Wallis and Futuna 🇹🇴 Tonga 🇦🇬 Antigua and Barbuda 🇨🇽 Christmas Island 🇭🇲 Heard Island and McDonald Islands 🇫🇴 Faroe Islands 🇬🇸 South Georgia and the South Sandwich Islands 🇰🇾 Cayman Islands 🇼🇸 Samoa 🇵🇲 Saint Pierre and Miquelon 🇫🇲 Micronesia, Federated States of 🇲🇻 Maldives 🇰🇮 Kiribati 🇳🇫 Norfolk Island 🇧🇻 Bouvet Island 🇰🇳 Saint Kitts and Nevis 🇬🇫 French Guiana 🇵🇫 French Polynesia 🇹🇻 Tuvalu 🇸🇹 Sao Tome and Principe 🇲🇭 Marshall Islands 🇻🇨 Saint Vincent and the Grenadines 🇳🇺 Niue 🇬🇬 Guernsey 🇷🇪 Réunion 🇮🇲 Isle of Man 🇨🇨 Cocos (Keeling) Islands 🇩🇲 Dominica 🇧🇶 Bonaire, Sint Eustatius and Saba 🇲🇹 Malta 🇬🇮 Gibraltar 🇮🇴 British Indian Ocean Territory 🇺🇲 United States Minor Outlying Islands 🇦🇼 Aruba 🇳🇷 Nauru 🇬🇩 Grenada 🇬🇵 Guadeloupe 🇸🇽 Sint Maarten (Dutch part) 🇱🇨 Saint Lucia 🇧🇭 Bahrain 🇲🇵 Northern Mariana Islands 🇦🇮 Anguilla 🇸🇨 Seychelles 🇱🇮 Liechtenstein 🇲🇸 Montserrat 🇦🇽 Åland Islands 🇸🇬 Singapore 🇻🇦 Holy See (Vatican City State) 🇹🇨 Turks and Caicos Islands 🇧🇱 Saint Barthélemy 🇦🇩 Andorra

[cypressf]

Many of the countries that are missing are possibly map units (smaller units than countries) https://www.naturalearthdata.com/downloads/10m-cultural-vectors/10m-admin-0-details/

See https://peakbagger.com/pbgeog/countries.aspx#depentab

[cypressf]

For now I'm going to add the sovereign nations in the map to the database, and ignore the countries listed above. Next, I'll investigate how to add them to the map.

[cypressf]

Frontend architecture

1. Request map_visualizations with param geography_type (values: 1: usa-county,2: country). If geography_type is set to neither or both it will include all map_visualizations, but if it is set to one or the other, it will only return items of that type.
2. If any data tabs are empty given the current filter, do not display them.
3. The geography_type param will be set by a tab or a radio toggle for now. It should persist in the redux store, but doesn't need to appear in the url for now.

[cypressf]

We're planning on sticking with ISO standard 3166-1 numeric country codes as of 2022.

[cypressf]

I'm resuming work on this now that I've released county report cards

[cypressf]

I'm running into an issue where Table<MapVisualization>.all(true, None) returns duplicate map visualizations and doesn't return any that are not in a tab. The SQL query needs experimenting with and I need to write some tests for it.

[cypressf]

Fixed that issue!

[cypressf]

Now I'm seeing a crash when I toggle to Global view. I'll investigate tomorrow.

[cypressf]

It was crashing because there were no world maps for the selected tab when I toggled from USA to world. The solution is twofold: pass an empty object instead of undefined as the state to components that expect map visualizations, and automatically filter tabs out if they are empty, and select the first non-empty tab.

[cypressf]

I had to modify the data processor so it could handle country codes because it had assumed that all data was index by usa-county SIPS code: https://github.com/mit-jp/climate-risk-map/commit/aacee71f7bb1d4366397ffe4aa0479931adfc9e7

[cypressf]

It might make more sense to add geography_type to dataset instead of map_visualization because each dataset is inherently associated with either the world or the USA. Someone can't modify a map visualization to be of the World if geographic region of the dataset is the USA

[cypressf]

• [x] bug: if you make a change to a map in /editor, the map selection list selects the first map, even if you're viewing a map lower in the list
• [x] bug: the global map projection is wrong (upside-down and small)
• [x] bug: the global map does not color the countries
• [x] one failing acceptance test

[cypressf]

It's looking like it'd be easier to use one table for all data, and make the primary key a composite of the id column and a geography_type column. That way, we wouldn't have to make a separate queries for every geography type. Instead, it would be one query, with the geography type specified in the query. For state-filters, however, we'd need to filter by id prefix since we could no longer have the id in the query itself.

[cypressf]

Okay. Global maps are ready to go. I just need to add data.