search

openclimatefix / satflow

Satellite Optical Flow with machine learning models
https://satflow.readthedocs.io/en/stable/
MIT License
61 stars 10 forks source link

Train Model on Optimal Cloud Analysis product #3

Open jacobbieker opened 3 years ago

jacobbieker commented 3 years ago

DARDAR has some very detailed maps from CALIPSO and CloudSat satellites. Unfortunately, the data ends around 2016-2017. There could be some use in still training a model on historical data, then using it to predict labels on newer clouds? It includes detailed breakdowns of the types of clouds as well, compared to most cloud masks are binary cloud/no-cloud without considering the different between a thin cloud blocking only some irradiance, and a thick cloud blocking essentially all of it for example.

jacobbieker commented 3 years ago

While actually using DARDAR doesn't seem very feasible for making a model, this product: https://navigator.eumetsat.int/product/EO:EUM:DAT:MSG:OCA was calibrated off DARDAR and gives more detailed cloud properties, but is only every hour. But could give something similar?

jacobbieker commented 3 years ago

Also could be useful are these products: https://data.eumetsat.int/product/EO:EUM:DAT:METOP:IASIL1C-ALL?query=IASI%20L1C&s=extended and https://data.eumetsat.int/product/EO:EUM:DAT:METOP:AVHRRL1?query=&s=simple that give more detailed cloud and data streams. So in training we could give higher resolution masks, or more cloud properties, and then have it predict those along with the images in the future? Would probably need to mask the loss from the areas of the image not covered by these products, as these are not geo satellites.

jacobbieker commented 3 years ago

Also, these products have the advantage of being from the product store, so we can download them without needing to wait some indeterminate time, like the other product.

jacobbieker commented 3 years ago

Try it with https://github.com/facebookresearch/xcit which is available from timm