Mock up dashboard that shows relevant spatial data layers

[angelchen7]

Summary

At the last meeting for the Delta Restoration group, they were thinking about potential tasks for me to work on for the rest of August and thought it might be good for me to create a mock dashboard that can access and show the spatial data layers in this spreadsheet. I think Eric Holmes might have started a mock dashboard for this already but I'm not sure if he intends to work in this one since it was updated a long 4 months ago. I will need to touch base with him first to see if he's started a new one or wants to work in the dashboard that he already has.

Edit: the more I look at this, the more I think that a new repo for the dashboard seems more appropriate.

Starting Tasks

• [x] Get clarification on exactly which layers they want
• [x] Search around to see where/how to access those layers

Useful links

• Spreadsheet of data layers that need to be accessed through the dashboard
• Eric's repo for his mock dashboard
• Learning Hub's flexdashboard lesson
• restoration_relevant_layers folder on Box
• potentially useful Urban Heat Island Index metadata

[angelchen7]

Found and downloaded all the layers that I could

I updated the spreadsheet to reflect all the datasets that I was able to find and download to Box. Some of them had links that were pretty hard to find but I got most of them (except for the two low-priority datasets). All the uploaded datasets now live in the restoration_relevant_layers folder on Box.

[angelchen7]

Restoration Meeting Updates

We had our biweekly Restoration group meeting today and I told them what I had done so far. After gathering all the data layers, here are some next steps:

• Try to get Eric's newest script for the flexdashboard to work (Delta_flexshiny_GPC_SVI+CES.Rmd, see email)
• Play around with visualizing the layers
• Eric also talked about needing help extracting data at the centroids

[angelchen7]

Visualizing the layers (and a curious situation)

I got Eric's script to work and I visualized a couple of the layers so far. All layer updates are tracked in the spreadsheet.

Right now I'm working on plotting the Urban Heat Island Index dataset and it's a bit weird since the dataset is split into many separate shapefiles so I have to hunt down the parts I want. Each part covers a certain region in California and for some reason, the East Bay shapefile is overlapping with the Antioch shapefile. The overlapped shapes have conflicting Urban Heat Island Index scores.

For example, for this shape (OBJECTID 468), the East Bay shapefile is saying that the Index is 31796.7 while the Antioch shapefile is saying that the Index is 2492.06. I'm not sure how to interpret this. Might be worth bringing it up to the group.

[angelchen7]

Restoration Meeting Updates

I showed the group the situation above and we basically agreed to skip over this dataset for now since it seemed like they calculated the UHII relative to each region and that wasn't too helpful to the group.

Other than that, I've been adding the layers one-by-one to the dashboard and it's going good so far.

Also, Eric wants me to extract the data at these centroids/points afterwards: Project coordinates: "\Box\NCEAS - Restoration\data\composite_data\step1_lookup.csv"

[angelchen7]

All possible layers added to dashboard!

I added all the layers I could and here are some screenshots of what each layer looks like now.

I drafted an email that I could send to the group next week to let them know what I did and if we should make a new repo for this dashboard.

[angelchen7]

Next steps

The group met yesterday (I had to teach the SSECR course so I couldn't make it) and discussed some next steps for me. The group detailed a few next steps for you to do for the Flexdashboard and analyses:

1. Integrate the CDC SOVI Layer into the dashboard, it should be located: "\Box\NCEAS - Restoration\data\sandbox\SOVI\CA_SOVI_theme_1.shp”
2. Convert the restoration project polygons into coordinate points using their centroid so that all the sites look the same
3. Extract the CDC SOVI values and join them to the restoration coordinates
4. Extract the values from all the other layers and join them with the restoration coordinates
5. If you have time, start optimizing performance of the dashboard so that it loads quicker (e.g. condensing files, scaling up the resolution, clipping the layers to the Bay-Delta extent, etc.)

[angelchen7]

Finished extraction and optimizing

First I switched out the original restoration sites with Eric's centroids so that all sites are now points instead of a mix of points and polygons. Then I extracted values from all layers and join them with the restoration points using st_join().

Finally I had to deal with the resulting bloated html file size. It was around 220MB and GitHub Pages only supports <100MB. To bring down the file size, I clipped the layers to the exact study boundary since I wasn't doing that before. Nevertheless, the html file was still pretty big (~190MB).

Then I was experimenting with converting the flexdashboard to a Quarto dashboard instead. However, it turns out that the Quarto dashboard html is still roughly the same size. I like the overlook look of the Quarto dashboard more, though, so maybe I'll make a pitch to the group to convert to Quarto...

I checked the memory that each R object was taking up and it turns out that the NOAA flood hazard data was really big in comparison to the other data layers. I then got the idea to downgrade the resolution so the sf object can be simplified and take up less space.

To do that, I used st_simplify() with the argument dTolerance = 0.0001 where it's in decimal degrees. I get this warning: "Warning: st_simplify does not correctly simplify longitude/latitude data, dTolerance needs to be in decimal degrees". I think it's fine to ignore this warning for now since I checked and compared the simplification and it seems to do the job good enough (for visualization purposes).

Before simplifying:

After:

As a result, the html file went from 190MB to 54MB 🎉 !!

According to this StackOverflow, the warning will go away if we use a projected CRS.

Helpful link on simplifying shapes with sf

[angelchen7]

Demo site is live 🎉 !! And potential next steps

Demo site: https://angelchen7.github.io/delta_map/

GitHub repo: https://github.com/angelchen7/delta_map

I decided to deploy a demo site for the dashboard because it would be way easier to show the progress to the whole group this way (instead of screensharing through Zoom). I emailed the group to let them know the URL for now and it seems like Alex has ideas for some next steps (see below).

Angel: since some of the layers are points or polygons that may not overlap with the restoration project points, I have some ideas on how to use them and what data to extract to the project points. Everyone else, feel free to jump in with alternative ideas or let me know if these don't make sense!

1. Parks with public access (Trust for Public Land layer). Can we calculate a layer based on this like "% park area in tract"? Where we calculate, for each census tract, park area in the tract divided by total area in the same tract, represented as a percentage.
2. CPAD and CCED layers (protected areas and easements). Can we calculate a similar layer to the one above, like "% protected area in tract"? Where each census tract gets the tract area in a protected area (CPAD) OR easement (CCED) divided by the total area in the tract, represented as a percentage. I'm proposing to have separate layers for #/1 and #/2 because I think #/1 may be a better predictor for projects providing cultural services (e.g. recreation, because the layer should be publicly accessible parks only) and #/2 may be a better predictor for projects providing supporting services (e.g. habitat improvement, because the layer represents protected lands that aren't necessarily open to the public).
3. Cleanup sites. It looked from the screenshots like you added the point layer from EnviroStor to the flexdashboard. Could we instead use the CalEnviroScreen layer that's calculated based on the sites, which are the percentile results? (the red polygons below – link to this map here. I think you can download data from this page)