Drexel-UHC / salurbal-city-selection-scrolly

Contains codebase and website deployment for SALURBAL city selection scrolly
https://drexel-uhc.github.io/salurbal-city-selection-scrolly/
0 stars 2 forks source link

Scrollytelling: SALURBAL City Selection #1

Closed ran-codes closed 5 months ago

ran-codes commented 2 years ago

Pending Issues

Action Items

Discussed in https://github.com/Drexel-UHC/salurbal-project-dashboard/discussions/101

Originally posted by **usamabilal** February 3, 2022 Following OLS idea of putting details on city selection, I think a great idea would be to create a "visual essay" summarizing it. Examples provided by @rl627 in another thread: From google's ML group: https://pair.withgoogle.com/explorables/hidden-bias/ from a 'visual essay' news outlet: https://pudding.cool/2017/01/shape-of-slavery/ undergraduate major story: https://cuthchow.github.io/college-majors-visualisation/ My idea for this would be: 1. How did we select cities? Brief text explaining AUE and citypopulation.de and selecting cities based on >100k on 2010. 2. Zoom into a city "core" (e.g. CABA in Buenos Aires). Show satellite imagery. Something about how this is the city core of buenos aires with 3 million people etc. The city itself is just those 3 million people. 3. Zoom out to the entire urban extent of the metropolitan area. Something about how this has 16 million people, and represents a continuous urban extent. 4. Overlay L2s, and "draw" (manually) the urban extent. Say something about how we selected local administrative units that overlap with this. 5. Shade somehow those that end up being part of buenos aires L1AD. Say these are the LAUs that are part of the L1AD. 6. "Merge" all L2s into the L1AD, and say "This is what we call Level 1, the city". Can copy text from brief on what L1 is. 7. "Zoom in again" and show L2s again. Say these we call L2s, subcities. Copy text from brief 8. "Zoom in again" and show L3s. Say these we call L3s, neighborhoods. Copy text from brief 9. Zoom out to the entire region, and just have a dot per city, and say we repeated this process for the 371 cities in x, y and z. (all feedback is welcome!) useful resources: - data brief (lots of useful text): https://drexel.edu/~/media/Files/lac/Data/Data%20Brief_ENG_FINAL_web.ashx?la=en. this is also translated into Spanish and Portuguese in case we wanted the essay to be translated too - the infamous city selection document: https://drive.google.com/file/d/1LatY_LP5VlpJWZL8VjdiBWIJK_iqPNr1/view

Resources dump

Draft v1

Draft v2

ran-codes commented 2 years ago

react-scrollama

react-scrollama has compatablity issues with React 16/18 (see open issue. For now we will just use vanilla js scrollama to prototype then refactor to react-scrollama when available. A few things to note (for future reference) when using vanilla javascript in react world:

Vanilla JS can interact with the DOM, but in React we are working with a virtual DOM so we cannot use traditional selectors (such as getElementById or getElementByClass). WE can use the useRef() React hooks to select elements in the virtual DOM and do do DOM manipulation or event observations as in normal JS.

scrollama.js

Utilizing useRef to get access to DOM elements. I implemented the regular JS version of Scrollama's stick-side template (ref). Of note, is that the scrollama() initialization uses traditional JS element selections (not classNames) so we kept classes instead of classNames in the JSX; the implication here is that styling is done through classes and thus CSS has been added to globals.css accessible to the entire application. This shouldn't be an issue as we use localized CSS modules for all other react components.

Satilite Imagery Implementation

Either google maps or leaflet works. GOing to start with leaflet as I have experience and it is open source (no API key required). Here is a tutorial for using leaflet + D3 together (http://bl.ocks.org/d3noob/9211665); here is a similar example implementing it all in React (https://stackoverflow.com/questions/66179826/how-to-display-json-in-d3-over-leaflet-js-in-react).

Boundary transitions

We pass the scroll state into our leaflet component. Here we need to do layer control based on scroll state. Leaflet bible here. One way to control transitions is to assign refs to each boundary layer then add/remove to map based on scroll state.

ran-codes commented 2 years ago

URL: https://wonderful-moss-0f45f310f-109.eastus2.azurestaticapps.net/city-selection

@usamabilal draft is up! I made the "About" button on the nav-bar as a dropdown and nested this page along with 'about'. Can always change location of this page later. Note still waiting on L3 geojson so that one will be filled in once i get the files. Let me know what you think.

usamabilal commented 2 years ago

this is lovely!!! such a great thing!! What is pending for the L3 geojsons? For the last step, how feasible is it to:

Other than that, this looks exactly how I was envisioning it. I think once we have the L3s we can circulate for feedback. @sjmelly and @kam642 what do you think?

ran-codes commented 2 years ago

What is pending for the L3 geojsons?

@sjmelly is compiling them into geojson format.

For the last step, how feasible is it to:

Make it so that clicking on a dot zooms into the city and shows L2/L1AD/L1UX

Should be straightforward. I think you suggestion is the logical next step: to recycle the components developed for Buenos Aires and just feed other city specific data to do scrollytelling of city selection for each city. That way we can provide details for each city.

These city specific city selection scrolly can then be linked in many ways:

Or a tooltip with the name of the city

Yes definitely can add hover-over tooltips for city name on top of circle markers.

sjmelly commented 2 years ago

We had an exchange that proposed focusing on examples of L3s for selected cities. I'm wondering if it makes sense to focus on L3s in Buenos Aires & Brazil cities when we are planning to publish small area estimates for L 2.5s. There are over 240,000 L3s. At their scale we probably do not want to simplify the boundaries, or at least not by much.

Can you work with ArcGIS web layers? AR L3s (Argentina_SALID3_5_4_21) can be found at https://services.arcgis.com/707Mk8fFdkSwTkI9/arcgis/rest/services/Argentina_SALID3_5_4_21/FeatureServer AR 2.5s (AR_SEC_ENE_sALID2_5_10_5_21) can be found at https://services.arcgis.com/707Mk8fFdkSwTkI9/arcgis/rest/services/AR_SEC_ENE_SALID2_5_10_5_21/FeatureServer

sjmelly commented 2 years ago

I jumped the gun and wrote down some comments on the content SALURBALLevelDefinitionnotes_2_18_22.docx

andreabolinaga commented 2 years ago

Hi all! I'm still new to this, but thank you for including me. This looks great and very interesting. I saw that Steven proposed a few edits on the text, I will add that the third line needs to be edited as well: I would change it to: " You can scroll to see an example of this process for the city of Buenos Aires" Also, would this have a title? Bare with me if these are obvious questions!

ran-codes commented 2 years ago

L3 Layers @sjmelly

We had an exchange that proposed focusing on examples of L3s for selected cities. I'm wondering if it makes sense to focus on L3s in Buenos Aires & Brazil cities when we are planning to publish small area estimates for L 2.5s.

My intuition is that we can present both L2.5 and L3 for cities that have them; in other words if a city only has L3 data we don't present L2.5 but if they have both then we present both. @usamabilal what do you think?

There are over 240,000 L3s. At their scale we probably do not want to simplify the boundaries, or at least not by much.

Unsimplified boundaries should work!

Can you work with ArcGIS web layers?

Haven't worked with ArcGIS API before. Would it be possible to save them on the UHC server somewhere in /Dashbaords/Spatial ?

usamabilal commented 2 years ago

On L3/L2.5 I'll defer to Steve/DMC. I agree wtih Steve that it may be better to show a "cleaner" city in that sense. Maybe Santiago? It has a small central L2, a very large L1AD/L1UX, and "pure" L3s

sjmelly commented 2 years ago

Ana Diez Roux & I were talking about the Brazil and Argentina small area estimates, and she thinks for those countries we should use the L2.5s as "de facto L3s" for people outside of the DMC.

ran-codes commented 2 years ago

@sjmelly sounds good! Then for now, for all Brazil and Argentina cities will display L2.5 and not L3. Will make this change soon.

Are L2.5 available for all the countries?

sjmelly commented 2 years ago

No. This raises the question of whether we should create de facto L3 datasets that include L3 measures for most countries and L2.5 measures for Brazil, Argentina and Costa Rica.

sjmelly commented 2 years ago

I liked the idea of using Santiago as an example, until I remembered that Chile was one of the countries where the L3s are not fully nested in L2s and we have L3SALTYPE = L3AD and L3SAL

ran-codes commented 2 years ago

react-scrollama

Note that react-scrollama team has addressed the bug we found (https://github.com/jsonkao/react-scrollama/issues/74). We can now try the react-scrollama.js implementation rather than scrollama.js

L2.5/L3 geojson or geodatabase

@sjmelly Any updates on the L3/L2.5 geojson files? I think we agreed on using L2.5 for Brazil and Argentina and then L3 for other countries (from above). If you want I can pull from the ArcGis geodatabase as well, just not sure where the files are located.

sjmelly commented 2 years ago

Do we want one city as an example of L3/L2.5? Santiago? Do we want to show just the L3AD based on administrative boundaries or include the L3SAL that we created in less urban areas where L3s from an authoritative source were not available. It seems to me the L3SAL are hard to explain to people. We will not have life expectancies or survey data for L3SAL.

ran-codes commented 2 years ago

Sure I will make a version for Santiago! As for L3SAL vs L3AD, what ever you think is best!

sjmelly commented 2 years ago

I saved a geojson of Santiago L3AD with topology corrected to \Projects\Wellcome_Trust\Data Methods Core\Dashboards\Spatial\geojsons\Santiago_L3AD_3_25_22.geojson

ran-codes commented 2 years ago

Thanks, @sjmelly! I will start working on recycling the scrolly to Santiago. 😄

In the meantime, can you provide the other cities' L2.5 and L3 shapes as geojson? Perhaps two geojson: with one containing all available L3 and another containing all available L2.5?

ran-codes commented 1 year ago

Stale issue. Labeled as On-hold will revisit later.

ran-codes commented 1 year ago

@usamabilal, the city selection scrolly is ready for review.

usamabilal commented 1 year ago

Some edits from me added there. I also added a comment for @sjmelly (on the public availability of the arcgis online map). Email to @kam642 and @kindvik sent for sharing with DMC and policy group for feedback.

usamabilal commented 1 year ago

@usamabilal to test accessibility WaVe has some errors https://wave.webaim.org/report#/https://data.lacurbanhealth.org/tools/city-selection I checked deuteranopia, protanopia, and tritanopia and the L3 boundaries are visible in all. pilestone (1)

kam642 commented 1 year ago

@ran-codes @usamabilal @andreabolinaga We were looking at the city scrolly at the internal DMC meeting today and Ana and @aquistbe have some major concerns. They think that this is too simplified and a lot of the context of our "cities" not being real defined cities is lost. And there is too much emphasis on the urban area being what Daniel's group created for urban area not what we (Alex) really did in our "eyeball" look to determine which admin units were grouped together.

She reviewed a document last year sometime (I'm attaching @aquistbe version here - we don't that the one that went to Ana I believe through @andreabolinaga but I could be wrong on that. I'm attaching.)

city_selection_scrolly ub_daq.docx

She is worried that a lot of the intro parts of this are lost that gives a lot of context. Was there a reason this was removed? Is there going to be a link to something that gives more details on the city selection process?

Also, she does not want to use Buenos Aires because of the complicated nature that it uses different units for L2 and it's causing unneeded confusion.

Scrolling up on this feed, I see there is a post on Mar 24, 2022 saying he was going to switch out to Santiago. Is there a reason that didn't happen?

Sorry for the long post!

usamabilal commented 1 year ago

Thanks Kari. Continuing this over email to include ADR

usamabilal commented 12 months ago

Starting to work on this again. One question for @sjmelly: what would be a good city to replace BA for? (to avoid multiple unit names, etc.). We need one with complete coverage of L3s. Can't be Santiago because of the zona censales issue (right? Santiago is not fully covered by L3s). Maybe Sao Paulo? note that we want to use ACTUAL L3s, not L2.5s.

Current document, incorporating most of the feedback is here: https://drexel0-my.sharepoint.com/:w:/g/personal/rl627_drexel_edu/EaY6i3RhGCpCirFZhwebqPMBz_cbJ5iAk3WEZqTv1xUaew?e=0DBPsb (i'll incorporate an extra few things people sent over email)

@ran-codes , any chance we can have this by Monday?

sjmelly commented 11 months ago

Previously we thought Santiago would be a good city in spite of not having fully nested L3s. For BR and AR which have fully nested L3s we are using L2.5s. GT also has fully nested L3s but we are using 2002 boundaries

ran-codes commented 10 months ago

reviewing the Word doc from Katie. A few quick questions/requests for @usamabilal or @aquistbe or @sjmelly

There are two cases where I need specific city names for:

  1. In cases where the administrative units that compose a city are very large, a SALURBAL city may include some areas that are not built-up or urbanized. what city is an example of this?

  2. early half of SALURBAL cities include only one administrative unit. what city is a good example of this.?

sjmelly commented 10 months ago

Mexico cities are good examples of cities with large areas that are not built up. I created a bookmark for Monterrey in https://drx.maps.arcgis.com/apps/mapviewer/index.html?webmap=99f47ed87f39402caee0e04318dbc2b6 as an example. For cities with only one administrative unit I can't think of what would make a good example over other choices. If you want to stay in Mexico, Nuevo Laredo and Victoria near Monterrey are examples.

aquistbe commented 10 months ago

Other places that have cities with a lot of non-developed areas would be in Argentina where the L2s are typically also larger, perhaps some in Chile, too.

I think that cities with one L2 typically were cities with lower populations, closer to the 100k minimum. I can’t think of specific ones that come to mind either at the moment, but also MX and AR probably have more of those due to larger L2s.

Best,

Alex

-- Alex Quistberg, PhD, MPH (he/him/his) Assistant Research Professor Director of Urban Health MPH Program

Urban Health Collaborative Dept Environmental & Occupational Health Dornsife School of Public Health Drexel University 3600 Market St, 7th Flr Philadelphia, PA 19104 USA 267.359.6297 | @aquistbe

Statistical Editor, Injury Prevention [signature_225253728]

From: Steve Melly @.> Date: Friday, August 18, 2023 at 10:44 AM To: Drexel-UHC/salurbal-project-dashboard @.> Cc: Quistberg,Alex @.>, Mention @.> Subject: Re: [Drexel-UHC/salurbal-project-dashboard] Scrollytelling: SALURBAL City Selection (Issue Drexel-UHC/salurbal-city-selection-scrolly#1)

External.

Mexico cities are good examples of cities with large areas that are not built up. I created a bookmark for Monterrey in https://drx.maps.arcgis.com/apps/mapviewer/index.html?webmap=99f47ed87f39402caee0e04318dbc2b6 as an example. For cities with only one administrative unit I can't think of what would make a good example over other choices. If you want to stay in Mexico, Nuevo Laredo and Victoria near Monterrey are examples.

— Reply to this email directly, view it on GitHubhttps://github.com/Drexel-UHC/salurbal-city-selection-scrolly/issues/1, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACAW2SZRNNMLPSNGBS5KHFDXV6EXPANCNFSM5N56R72A. You are receiving this because you were mentioned.Message ID: @.***>

usamabilal commented 10 months ago

Cuauhtemoc in Chihuahua has the lowest pop density of all cities with 1 L1AD.

image

(the city is just that tiny thing at the bottom). SALID1 is 204110

usamabilal commented 10 months ago

Rio Cuarto in Cordoba (AR) same thing:

image

SALID1 is 101108

ran-codes commented 10 months ago

Great insights. Thanks everyone for timely replies = )

ran-codes commented 10 months ago

@usamabilal First draft done and can be view here. I made this based on the following word doc from Katie: Scrolly - Salurbal City Selection-6 11Aug2023.docx.

Feel free to update the word doc or send a check list of edits/changes! have a great weekend!

kindvik commented 10 months ago

Hi Ran, I think it's looking great!! I will send a few minor comments next week before circulating for external feedback as discussed. Thank you for all of your work on this! Katy

kindvik commented 10 months ago

Never mind...no time like the present! Please see attached with a few edits and comments. Thanks again!! Katy Scrolly-Salurbal.City.Selection--25aug2023kbi.docx

ran-codes commented 10 months ago

Thanks for the comments/feedback @kindvik.

Updates have been made please see here: https://drexel-uhc.github.io/salurbal-city-selection-scrolly/.

(Please hit cntl+f5 to hard refresh the page if you don't see updates ... sometimes github pages take a second to refresh after deployment).

sjmelly commented 10 months ago

I added some comments to Katie's Scrolly-Salurbal.City.Selection--25aug2023kbiSM.docx

ran-codes commented 10 months ago

@kindvik can you triage steves comments? (I'll wait for any further changes from you). Thanks!!

kindvik commented 10 months ago

Hi all,

Thank you, Steve, for taking another careful look at this!

Ran, please see my responses/suggestions attached. I think all but the comment at the end of page 2 are easily resolvable - maybe @@.> or @@.> could comment?

Thanks again, I think it's looking great!

Katy

ran-codes commented 10 months ago

@kindvik thanks!! (don't see anything attached in the email ... maybe I missed it 😢 ). can you reattach and send thanks!

kindvik commented 10 months ago

Hi Ran, does this work? Thanks!! Scrolly-Salurbal.City.Selection--25aug2023kbiSMkbi2.docx

ran-codes commented 10 months ago

perfect. thanks!

ran-codes commented 10 months ago

@kindvik updates have been made.

Circle size @sjmelly

Does this look better to you?

image

ran-codes commented 10 months ago

forgot to upload the word doc. please see: Scrolly-Salurbal.City.Selection--25aug2023kbiSMkbiRL3.docx

sjmelly commented 10 months ago
ran-codes commented 10 months ago

Thanks for your comments @sjmelly. updates have been made accordingly.

@usamabilal happy to hear you comments.

aquistbe commented 10 months ago

A few comments:

Does it make sense to have “Steps” still when there only 2? If we eliminated that, we could perhaps ignore my comments below about moving some of the sentences to Step 2 from Step 1 (though my suggested rephrasing would still stand).

Step 1: where it says, “eliminated overlaps,” I think it would be better to say “eliminated duplicates” since the former makes it sound like to me we eliminated the cities they had in common.

Step 1: starting where it says, “Cities that were very close together and were therefore part of the same urban agglomeration were combined” should go to Step 2 since it was part of that process. I would also rephrase it, “Cities that were initially considered separate cities were combined into one city if they were part of the same agglomeration, i.e., their built-up areas connected. For cities connected in the same urban agglomeration, we assigned it the name of the city with the larger population, but if the populations were nearly equal we assigned a hyphenated name.”

Step 1: “We could keep this here, but should perhaps note that the final list of cities came later after we had checked all the built-up areas and assembled the L2s.

Best,

Alex

-- Alex Quistberg, PhD, MPH (he/him/his) Assistant Research Professor Director of Urban Health MPH Program

Urban Health Collaborative Dept Environmental & Occupational Health Dornsife School of Public Health Drexel University 3600 Market St, 7th Flr Philadelphia, PA 19104 USA 267.359.6297 | @aquistbe

Statistical Editor, Injury Prevention [signature_3851257108]

From: Ran Li @.> Date: Monday, August 28, 2023 at 1:27 PM To: Drexel-UHC/salurbal-city-selection-scrolly @.> Cc: Quistberg,Alex @.>, Mention @.> Subject: Re: [Drexel-UHC/salurbal-city-selection-scrolly] Scrollytelling: SALURBAL City Selection (Issue Drexel-UHC/salurbal-city-selection-scrolly#1)

External.

Thanks for your comments @sjmellyhttps://github.com/sjmelly. updates have been made accordingly.

@usamabilalhttps://github.com/usamabilal happy to hear you comments.

— Reply to this email directly, view it on GitHubhttps://github.com/Drexel-UHC/salurbal-city-selection-scrolly/issues/1#issuecomment-1696147853, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACAW2SYZWTR5SU3RDI6OHI3XXTPH5ANCNFSM6AAAAAA3Y4XRQA. You are receiving this because you were mentioned.Message ID: @.***>

kindvik commented 10 months ago

Thanks, Alex. My two cents below in blue, with proposal for final text and changes highlighted; Ran, please let us know if you need in a different format, this was the easiest way for me to process quickly. Hopefully we are on the last round..!

From: Alex Quistberg @.> Sent: Monday, August 28, 2023 1:46 PM To: Drexel-UHC/salurbal-city-selection-scrolly @.> Cc: Indvik,Katherine @.>; Mention @.> Subject: Re: [Drexel-UHC/salurbal-city-selection-scrolly] Scrollytelling: SALURBAL City Selection (Issue Drexel-UHC/salurbal-city-selection-scrolly#1)

External. A few comments:

Does it make sense to have "Steps" still when there only 2? If we eliminated that, we could perhaps ignore my comments below about moving some of the sentences to Step 2 from Step 1 (though my suggested rephrasing would still stand). \ I think the steps are helpful for walking readers through the content; my suggestion below adds another for 4 total.

Step 1: where it says, "eliminated overlaps," I think it would be better to say "eliminated duplicates" since the former makes it sound like to me we eliminated the cities they had in common. Good point, see below.

Step 1: starting where it says, "Cities that were very close together and were therefore part of the same urban agglomeration were combined" should go to Step 2 since it was part of that process. I would also rephrase it, "Cities that were initially considered separate cities were combined into one city if they were part of the same agglomeration, i.e., their built-up areas connected. For cities connected in the same urban agglomeration, we assigned it the name of the city with the larger population, but if the populations were nearly equal we assigned a hyphenated name." See suggestion below; I added Vina/Valpo as an example of hyphenation.

Step 1: "We could keep this here, but should perhaps note that the final list of cities came later after we had checked all the built-up areas and assembled the L2s. I think it's okay as-is; this is inevitably a somewhat simplified version of the process, but I'll defer to @@.***> and others here.

Step 1. Identifying cities with a population of 100,000 or more. The SALURBAL city universe was defined as all urban agglomerations with at least 100,000 residents as of 2010. We used the Atlas of Urban Expansionhttp://atlasofurbanexpansion.org/ and country census data from citypopulation.dehttps://citypopulation.de/ to obtain a list of all cities (as defined in these sources) with 100,000 residents or more in 2010. We combined both lists and eliminated duplicates. Step 2. Combining neighboring cities. Cities that were initially considered separate cities were combined into one city if they were part of the same agglomeration - that is, if their built-up areas were connected within the same urban agglomeration. We used the name of the city with the larger population to label these units. In a few cases where the populations were nearly equal, we assigned a hyphenated name (e.g., Valparaiso-Viña del Mar, Chile). Step 3: Defining the geographic boundaries of each city.

We operationalized - or geographically defined - each SALURBAL city using existing administrative units to which health and other data could be easily linked. These administrative units included municipios, departamentos, or similar units in each country.

[...]

Step 4. Creating a hierarchy of geographic units within each SALURBAL city

usamabilal commented 10 months ago

Thanks @aquistbe . Added all your suggestions but kept the step language (we actually have 3!) as i feel it's useful. I understand what you mean in the consulting with countries,so i reorganized that part a bit (moved some of it down, some of it up). Document here: Scrolly-Salurbal.City.Selection-staging1.docx.docx

@kindvik FYI if you answer in email (instead of github directly) formatting is lost, so we dont see colors!

usamabilal commented 10 months ago

I took @kindvik and merged with my version above, doubling down on steps (we have 5 now!). Scrolly-Salurbal.City.Selection-staging ub.docx

Let me know if there's anything unclear Ran!