Create Greenest Pixel Composite

[cjthomas730]

@apericak you mentioned that a professor you spoke with recommended that we use Surface Reflectance products for our work, and that we will need to create our own Greenest Pixel Composites, especially for more recent years. Can you please summarize that conversation here, and work to create a script that will allow us to create our own GPCs? If you need any help, let me know.

[cjthomas730]

@apericak I've been playing around with this a little bit, which maybe we could talk about/work on during our call after the stand-up on 07/07?

[apericak]

Just for recording purposes, we will look at this on July 7; until then, I'm going to do some testing to see if we should use SR or TOA images, as that's the first step for doing the greenest composites.

[apericak]

Some thoughts as I look at this:

• The Googlers claim all they're doing to make their greenest composite is mapping Raw --> TOA, NDVI, and then qualityMosaic(). But mimicking that process yields a slightly different result for very cloudy areas than does using their premade greenest composite. So maybe they're not telling us something?
• In most cases, though, creating the greenest pixel composite ourselves (via TOA) is exactly the same as the premade composite. I have found some locations, though, where the premade composite still retains some error (near the edges of a path/row) that don't appear when making the composite myself. In short, we can probably just make the composite ourselves.
• Here's a script from Matt Hancher (of Google) about the extra cleaning Google uses to make the greenest pixel composites. We should definitely mimic this for our script (it's what I've been doing to test everything today).
• It looks like cloud masking the images ahead of time (via TOA) might actually make the output worse than not cloud masking. I think this happens because the cloud masking turns any cloud pixels--including thin clouds--into null values. So even if the cloud was thin enough that we could have gotten a real NDVI value from it, the cloud mask discards those. Notably, this is really only a problem where the greenest pixel composite nevertheless indicates clouds, but in these cases trying to mask the clouds actually makes them look larger. In most locations, though, there is no difference in greenest pixel values with or without cloud filtering, so it's probably best to not cloud filter.
• I think if we explain ourselves thoroughly, we should be alright with using TOA rather than SR. See this post in particular about why using SR might be a bad idea. We can justify using TOA in a few ways. 1) We get a wider range of NDVI values over mines with SR than with TOA, meaning our threshold will be less certain with SR. 2) We are correcting "over time" by making a composite of yearly images, and moreover by having that composite get maximum NDVI; there may not really be further gains to SR. 3) There isn't an easy way via GEE to get SR for the early Landsats (pre-LS4), so those are going to have to be TOA; keeping the later Landsats as TOA too will mean we are at least consistent with our processing for those.

In sum: let's use TOA imagery to make our own greenest composite, via the script above, but let's not worry about cloud filtering.

Still to test: should we exclude the top and bottom 5% of NDVI values from the compositing process, or some other additional processing to get rid of outlying values? And would that make a difference?

[cjthomas730]

Here's what I've been working on with greenest pixel composites: https://code.earthengine.google.com/ad586b085cd23b8dcf0e9fd07e948c1e

[apericak]

Update: I can't seem to find a difference by getting rid of the top/bottom 5% of values, so we can likely ignore that part too.

[cjthomas730]

@apericak so we are computing our own Greenest Pixel Composite in ndvi-workspace-analysis.js?

[apericak]

@cjthomas730 Right, but not in ndvi-workspace-vis.js, so that that one will run faster (and thus be better for visualizations, etc.) I'm open to renaming either or both file names, if that would make it clearer.

[cjthomas730]

@apericak alright, I'm going to close this issue for the GPC creation, and migrate the discussion about TOA v. SR to a new ticket.