search

lachlanhurst / observedearth-web

Website for ObservedEarth; an app for obtaining, processing, and visualising satellite imagery
MIT License
0 stars 0 forks source link

Filter out nighttime imagery #1

Open jflasher opened 8 years ago

jflasher commented 8 years ago

Didn't see a repo for the app code, so putting this suggestion here. The Landsat datasets has both nighttime and daytime imagery. Nighttime scenes aren't really great when showing things off visually, they lead to weird artifacts like http://landsat-pds.s3.amazonaws.com/L8/140/209/LC81402092016228LGN00/index.html. There is an open issue to add a day/night flag to https://s3-us-west-2.amazonaws.com/landsat-pds/scene_list.gz (if this is where you're getting your metadata), but for now, you could use cloudCover === -1 as a proxy for that. Filtering those out would keep visual artifacts from showing up.

lachlanhurst commented 8 years ago

Thanks @jflasher; both for this issue and introducing me to the landsat_ingestor repo. Here's the right spot for it; I have some thinking to do about opening up the source.

I did know about those weird scenes and had planned to remove them, but in the self-imposed rush to release the app forgot about it until reminded by another user the other day. I was however going to remove the affected WRS2 path/rows from the apps database, this would have meant no images from those path/row coordinates would be available (not ideal). Will use the cloudCover check as you suggest, the app already has this info available so will be an easy fix.

The app doesn't make use of the scene_list.gz nearly as much as I'd like. It's used to populate a database that gets packed into the app, but the app itself 'guesses' at what scenes may be available. Expect a feature request sometime soon ;-)

jflasher commented 8 years ago

👍 Having recently done something similar, I came up with the solution of running a Lambda function every 6 hours or so and creating a file per path that contains all the scenes contained within it. Then I just query that file (via a web endpoint) to figure out what scenes exist and their cloud cover and such. Basically it's taking that big scene_list and cutting it into smaller, useable pieces useable on the fly. Does introduce an external dependency, but gave me an updating mechanism.