Google Earth Engine as recipe destination

[simonff]

There are two ways to ingest raster data into Google Earth Engine:

• put COGs into a GCS bucket and point EE at them by creating a COG-based asset,
• put any geotiffs (not just COGs) into a GCS bucket and supply an optional manifest file to ingest them as an asset with tiles stored internally.

Note that even in the former case just producing the files is not enough, strictly speaking - the ingestion also requires an EE call to create an asset, though in most cases the call will be trivial. The latter case is useful because it does not need reprocessing to COGs and because it's generally faster to read assets with internal tiles.

If there is community interest, it would be nice to support EE as a recipe destination. That should lead to better code reuse.

[rabernat]

Hi @simonff and welcome! 👋

We discussed this question at yesterday's meeting. Everyone on the call was very happy at the prospect of interoperability between Pangeo Forge and GEE.

Of the two ways you enumerated, the first one is the most compatible with our goals. We have already planned to support COG outputs (see https://github.com/pangeo-forge/pangeo-forge/issues/63).

I think what is missing to move forward with this is a specific dataset to serve as a use case. If you have one on mind, please feel free to suggest it in https://github.com/pangeo-forge/staged-recipes/issues. The way we are steering development of Pangeo Forge is all around these use cases.

[simonff]

I don't have specific use cases yet, but this maybe a chicken-and-egg problem - EE users don't use pangeo-forge much, AFAIK, because they don't have a reason to if the EE destination is not supported.

I'd like to create a simple proof-of-concept example for a single-file dataset. Having COG support is 95% of the work.

The other, harder, issue is uploading vectors. Are there any plans for vector recipes?