search

mit-jp / mit-climate-data-viz

Plotting climate data for the MIT Joint Program on the Science and Policy of Global Change
https://cypressf.shinyapps.io/eppa-dashboard/
0 stars 0 forks source link

Try mapping overlay of roads #106

Closed cypressf closed 3 years ago

cypressf commented 3 years ago

Visual overlay available on every tab

cypressf commented 3 years ago

The roads shape file is too large to conveniently load in a web app right now. Even after reducing the size using tricks like stripping out all the metadata and converting to a topo file, it's still 17 MB, 25x larger than the US county map.

We may need to start with a less high-resolution road file.

cypressf commented 3 years ago

I'm having another issue with it—it's not lining up with the county map after using the same albers projection for both of them. I wonder if I made a mistake when converting them from shape file to geojson, or if the difference also exists in the original shape file.

Screen Shot 2021-02-26 at 15 12 38

cypressf commented 3 years ago

Since the map is too large to load quickly on the website, I'm looking for alternative less-detailed road maps. When I find one I'll test to see if it still has the same mismatched projection issue. If so, I'll look into it, if not, great—no need to debug now.

cypressf commented 3 years ago

Some examples of lower res data:

https://gist.github.com/bricedev/96d2113bd29f60780223 https://observablehq.com/@floledermann/drawing-maps-from-geodata-in-d3-condensed-version https://github.com/jgoodall/us-maps/tree/master/topojson-simplified

one got its road data from here https://www.naturalearthdata.com/features/. maybe we can get road data from there as well and simplify to topojson

cypressf commented 3 years ago

While I was researching roads I read a little about the Commodity Flow Survey, which the census says is the most detailed publicly available data on transportation of goods in the US. They have flow of goods from certain commodity areas, which are groupings of counties in metro areas. We could potentially map these on our county map along a visualization of flow between the areas.

Screen Shot 2021-02-26 at 15 33 03 Screen Shot 2021-02-26 at 15 33 27

cypressf commented 3 years ago

This is the source of the lower res road data in the gist linked above:

natural earth data: roads

side note: they also have

cypressf commented 3 years ago

I tried filtering the natural earth world roads data to just the usa, projecting it with albers, and plotted it on top of our counties. Screen Shot 2021-02-26 at 16 18 47

It's also offset! I think the projection doesn't line things up like I expect.

cypressf commented 3 years ago

The good news is this new file is only 1.7MB, and that's still with some metadata like road name and type. I think the size concern is mostly fixed. I just need to focus on getting the projection correct.

azzacasch commented 3 years ago

Hey Cypress,

Thanks for the update - this looks to be just the right level of detail with the roadways to start as a visual overlay and glad the file size shouldn't be an elephant! The projection is always the stinker with these things.

Have a great weekend, Adam

C. Adam Schlosser Senior Research Scientist, Center for Global Change Science Deputy Director, MIT Joint Program on Science and Policy of Global Change http://globalchange.mit.edu

50 Ames St. Building E19-439L Massachusetts Institute of Technology Cambridge, MA 02139-4307

Phone: +1-617-253-3983 Fax: +1-617-253-9845 casch@mit.edu

On Feb 26, 2021, at 4:29 PM, Cypress Frankenfeld notifications@github.com wrote:

The good news is this new file is only 1.7MB, and that's still with some metadata like road name and type. I think the size concern is mostly fixed. I just need to focus on getting the projection correct.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cypressf/mit-climate-data-viz/issues/106#issuecomment-786903641, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYYCPPYA3WBBQW773IHQE3TBAHCBANCNFSM4YBBPTHA.

azzacasch commented 3 years ago

Hey Cypress,

Just a quick response - this bit looks really promising! Looks to be the sort of thing we were talking about in terms of how to we somehow create a transportation “risk” metric so that it can be combined in our entry-level page.

I think I may have missed this when you originally sent it. I just discovered today that I’ve had some pathologic problem with my email (syncing to my laptop) and the only way I could really troubleshoot the problem was to remove my MIT Exchange account from my laptop and completely rebuild my inbox today. Arrrgh...

—a

C. Adam Schlosser Senior Research Scientist, Center for Global Change Science Deputy Director, MIT Joint Program on Science and Policy of Global Change http://globalchange.mit.edu

50 Ames St. Building E19-439L Massachusetts Institute of Technology Cambridge, MA 02139-4307

Phone: +1-617-253-3983 Fax: +1-617-253-9845 casch@mit.edu

On Feb 26, 2021, at 3:37 PM, Cypress Frankenfeld notifications@github.com wrote:

While I was researching I read a little about the Commodity Flow Survey https://www.census.gov/programs-surveys/cfs.html, which the census claims is the most detailed publicly available data on transportation of goods in the US. They have flow of goods from certain commodity areas, which are groupings of counties in metro areas. We could potentially map these on our county map along a visualization of flow between the areas.

https://user-images.githubusercontent.com/443397/109352254-83d9ce80-7848-11eb-9aab-dc4b6404f4a2.png https://user-images.githubusercontent.com/443397/109352264-85a39200-7848-11eb-940d-61558629fdef.png — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cypressf/mit-climate-data-viz/issues/106#issuecomment-786881275, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYYCPOJTGYERPVNQDV3CX3TBABCNANCNFSM4YBBPTHA.

cypressf commented 3 years ago

Ah ha! I figured out the issue. I forgot I had used an alternate, much more lightweight county map—not the one Alyssa originally provided me with. That county map used a slightly different projection than the one I was using to convert Alyssa's shape file to geojson. My head-banging is over.

d3.geoAlbersUsa().scale(1300).translate([487.5, 305]) instead of d3.geoAlbersUsa().fitSize([975, 610], d)

A nicely overlapping projection of roads onto the map

cypressf commented 3 years ago

Examples with data overlays: Screen Shot 2021-03-03 at 17 34 55 Screen Shot 2021-03-03 at 17 33 47

cypressf commented 3 years ago

Done in https://github.com/cypressf/climate-risk-map/commit/8b0a15cfaea7782ddeeccb301f77cf692e467437

azzacasch commented 3 years ago

Cool beans Cypress!

—a

C. Adam Schlosser Senior Research Scientist, Center for Global Change Science Deputy Director, MIT Joint Program on Science and Policy of Global Change http://globalchange.mit.edu

50 Ames St. Building E19-439L Massachusetts Institute of Technology Cambridge, MA 02139-4307

Phone: +1-617-253-3983 Fax: +1-617-253-9845 casch@mit.edu

On Mar 3, 2021, at 5:50 PM, Cypress Frankenfeld notifications@github.com wrote:

Closed #106 https://github.com/cypressf/mit-climate-data-viz/issues/106.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cypressf/mit-climate-data-viz/issues/106#event-4404695901, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACYYCPL7237AO5LZDDML4ZDTB24LRANCNFSM4YBBPTHA.