rasters for pollutant maps

[mmontesanonyc]

@cgettings @chuskey79 - in pollutant-maps-over-time I created buttons and a year slider, plus javascript that gets the values for the selected pollutant and year. if you open the console you can see that on clicking either set of controls, it logs to the console the variable polYear which is pollutant x year:

we can use this to tell the map which raster file to load (possibly via a json associating pollutant x year combo with a specific raster URL? or maybe we name them with a structure we can produce via the javascript).

@chuskey79 can you add a raster for black carbon 2022 so I can build the initial map?

[mmontesanonyc]

@cgettings - @chuskey79 asked me if he should provide the raster files, or geotiff. what do you recommend for using with leaflet?

[cgettings]

Probably both, and then we can test which one works better. But I'm inclined to say the GeoTIFF.

[chuskey79]

OK. I created two folders (Geotiff and Raster) in a new folder called RasterFiles with each.

From: Chris Gettings @.> Sent: Monday, November 21, 2022 10:39 AM To: nycehs/nyccas-vis-demo @.> Cc: Christopher Huskey @.>; Mention @.> Subject: [EXTERNAL] Re: [nycehs/nyccas-vis-demo] rasters for pollutant maps (Issue #1)

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Forward suspect email to @.**@.> as an attachment (Click the More button, then forward as attachment).

Probably both, and then we can test which one works better. But I'm inclined to say the GeoTIFF.

— Reply to this email directly, view it on GitHubhttps://github.com/nycehs/nyccas-vis-demo/issues/1#issuecomment-1322252332, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKDD3LDQDVE5BRSOIGV4RR3WJOJSRANCNFSM6AAAAAASFKEMDU. You are receiving this because you were mentioned.Message ID: @.***>

Sent from the New York City Department of Health & Mental Hygiene. This email and any files transmitted with it may contain confidential information and are intended solely for the use of the individual or entity to whom they are addressed. This footnote also confirms that this email message has been swept for the presence of computer viruses.

[chuskey79]

We don't have BC rasters, yet. I uploaded two NO rasters from a couple years back, since I didn't have to create new files.

[mmontesanonyc]

Sounds good, no prob - I just want to get something in there.

[mmontesanonyc]

hey @cgettings, I poked around with using georaster for leaflet but there's something about how it works that i don't quite understand.

if i use their load file example and load in one of @chuskey79's geotiffs, it displays just fine. So the file is good.

But when I followed georaster for leaflet's instructions to set it up in a local repo, and the page script that i'm writing throws an error around using require. I think it has to do with something that I don't quite understand about Node.js and how a project needs to be structured - pretty sure i need a separate js file? But I also forked the georaster-for-leaflet example and dropped in the URL for our geotiff and got errors. Anyway, would love to put our heads together next week - and wanted to type this up to remember what was up. Some potentially helpful stuff here that I'll pick at if I have time.

[cgettings]

Yeah, you can't use require unless you're building a Node module. So we need to use the usual html script src tag to load anything.

[cgettings]

This example might work.

[mmontesanonyc]

I also looked into this a bit, might be a little bit simpler? Will go deeper next week.

[mmontesanonyc]

basics are in place but it's throwing errors trying to load the geotiff.

the same file loads fine from the package's "load file" example, but when I modify their load-cog-from-script, i get errors trying to fetch the geotiff.

@cgettings do you have bandwidth to look at this? i'm banging my head against one wall after another.

[cgettings]

I'll try to take a look

[mmontesanonyc]

Thanks. No worries if you don't have the bandwidth but maybe it's easier to dig into small projects than big ones this week.

[cgettings]

What are the errors specifically?

[mmontesanonyc]

Uncaught (in promise) TypeError: Failed to execute 'fetch' on 'WorkerGlobalScope': Failed to parse URL from RasterFiles/Geotiff/noAA12final.tif at ie.blockSize [as retrievalFunction] at ie.requestData at ie.fetch at async xe.fromSource

[cgettings]

COG is a cloud optimized geotiff, which I don’t think our files are (yet). I don’t think that would throw a URL parse error, but it’s maybe worth looking into.

Any other fetch call differences?

[mmontesanonyc]

ah! ok. didn't realize that about COG. So, i grabbed a different example as a starting point, and now i've got a functional thing working. thanks!

[mmontesanonyc]

@chuskey79, I propose the following naming conventions:

For the last 3 years:

• focus-no2020.tif, focus-bc2020.tif', etc, using no, bc, no2, o3, pm, and so as the pollutant names, and 4-digit year.

For the all years archive:

• pollutant-year.tif, with no, bc, no2, o3, pm, and so as the pollutant names (same as above) and 4-digit year (also same as above).

And, per conversation, the "pollutants over time" part of the map will show the last 3 (?) years, scaled to mins and maxes within those years - and we can link to an "all years" archive that have a scale consistent across all years, so people can see the whole progress.

[chuskey79]

Put these files in github?

[mmontesanonyc]

Yup! You can plop 'em right in here.

[chuskey79]

I've started to put them here: https://github.com/nycehs/nyccas-vis-demo/tree/main/RasterFiles/AllYears

[mmontesanonyc]

@chuskey79, i'm trying to start fiddling with these, but when i try to map them, the console throws an error:

Is there something about these that differ from noAA12final.tif (which was functional)?

[chuskey79]

I think Sarah had exported the noAA12final.tif from her code, whereas I exported these from the raster files in arcgis. I wouldn't think that would cause an issue, but who knows...

[mmontesanonyc]

OK - if you could look into it that would be great! Also FYI I moved the rasters to a single location in /raster

[chuskey79]

It will be difficult to export the older files from sarah's code. I'll try to reexport and see what happens.

[chuskey79]

I changed the resolution to 200 dpi. I assume that wouldn't cause the issue, would it?

[mmontesanonyc]

I don't know. @cgettings might have insight here. pollutant-maps-over-time is throwing errors trying to map the geotifs, though an earlier file mapped just fine.

[chuskey79]

Looks like arcgis didn't write the GEO part of the GeoTiff because that would require a special license to do one of the basic functions of a GIS software product.

[chuskey79]

I'll have some tomorrow morning to test with before I redo the whole lot.

[mmontesanonyc]

Feel free to just do one - I can test and let you know if it works - before running off the rest.

[chuskey79]

Try no2009.tif in the raster directory.

[mmontesanonyc]

success! (ignore the buttons - that's no2009.tif).

[chuskey79]

ok. I'll start republishing in the morning. ArcGIS doesn't like dashes in file names, so we'll need to change the file naming convention slightly.

[mmontesanonyc]

Sounds good. We can just remove dashes from the convention we currently have in /raster. I deleted what was in there before. If you could put them all there, in the same folder, that'd be ideal.

[mmontesanonyc]

A few that don't work:

• bc2020, bc2016
• no2015, no2019
• no2 2009 through 2013

[chuskey79]

I think I know what the issue is. I'll republish today.

[chuskey79]

Hopefully, these work now.

When you have a chance, can you mock something up so that I can see how the rasters look? Normally I need to equalize the visable range on these and I'm wondering how that's going to work.

[mmontesanonyc]

yup, serve up pollution-maps-over-time.html and you should be able to check it all out.

[mmontesanonyc]

DECISION:

• 2021: rasters for each pollutant (internally scaled)
• All years: PNGs for each year/pollutant (named per approach above), consistently scaled.

Down the line: we'll do dynamic scaling & color based on raster values. (eg...)

[mmontesanonyc]

OK, pollutant-trend-images.html is working. Missing SO2 from 2017 to 2020, however. (And, I think we're waiting on 2021 data for everything.)

pollutant-maps is working, but it's set to show 2019. The 2021 files aren't working. Will troubleshoot.

[chuskey79]

I forgot to export the 2021 pngs. i'll do that in the morning.

I'm not sure I exported the geotiffs from arcmap pro, so that may be the issue. I'll reexport.

[chuskey79]

I uploaded the revised files. let me know if there are issues.

[mmontesanonyc]

For some reason, the 2021 rasters aren't working. Most of the previous years (with a few exceptions) are working fine. But the leaflet map isn't displaying the 2021 georasters. It throws an error about rendering the tiles. Are you exporting them the same way?

[chuskey79]

ugh

[chuskey79]

Try again....

[chuskey79]

We normally have a text element with the maps that describes the data for each pollutant.

Is is going to be possible to add this to the viz?

[chuskey79]

I uploaded a new geotiff file (no22021.tiff). I think it should keep the color rendering. Can you check it out and let me know? If so, I'll do the others...