Reproduce a simple EE script in C-Scale using OpenEO

[gena]

https://code.earthengine.google.com/d08354d7d4f0c1a1ec3178e5a4134feb

[avgils]

An online OpenEO editor is available: https://editor.openeo.org/

For testing you van use Earthengine as provider and login with group1, test123 https://editor.openeo.org/?server=https://earthengine.openeo.org/v1.0

[avgils]

A first simple notebook is added to the repository with an OpenEO NDWI example

[backeb]

@mariojmdavid @gdonvito please take note of some of the user requirements here

[backeb]

@gena @avgils an alternative solution to OpenEO that was put forward is: https://github.com/davidfrantz/force https://force-eo.readthedocs.io/en/latest/ @gena @avgils do you have any experience with FORCE?

FORCE is being used in @c-scale-community/use-case-return to access LANDSAT data.

@mariojmdavid and @gdonvito could you please indicate if you foresee any issues in making OpenEO available on your infrastructures?

[mariojmdavid]

hi all you can instantiate VMs as you see fit, I have put a quota of 3 public IPs, but use as less as possible, you will only need them for public exposed endpoints, any other VM can be reached through ssh tunnels from a single public IP you are free to setup the security groups as you want, opening the ports you need nonetheless we strongly advise that the endpoints are encrypted https/ssl, and when deploying endpoints not yet encrypted do not open them for the world, but only for your networks, we can help if needed

we may register domains in our DNS, and we are able to request x509 certificates for the endpoints if needed, if you do that on your side, we can as well just register reverse DNS on our side if needed

[sebastian-luna-valero]

Hi,

I would like to create a conda environment file to help reproduce the EE script. In theory this should streamline the deployment of the use case both in Cloud and HTC/HPC services.

Are all the dependencies listed in the links below required for the Aquamonitor use case?

• https://github.com/c-scale-community/use-case-aquamonitor/issues/7
• https://confluence.egi.eu/display/CSCALE/2021-07-08+T3.2+minutes

Namely:

• http://xarray.pydata.org/en/latest (especially for dealing with netcdf data)
• http://dask.readthedocs.io/en/latest/ (for parallel computing and scaling)
• http://jupyter.org/ (for interactive visualisation)
• https://openeo.org/ (for satellite imagery)
• https://scitools-iris.readthedocs.io/en/latest/ (nice to have)
• https://help.marine.copernicus.eu/en/articles/4796533-what-are-the-motu-client-motuclient-and-python-requirements (CMEMS MOTU client)
• https://gdal.org/
• http://nco.sourceforge.net/
• https://code.mpimet.mpg.de/projects/cdo/wiki/Cdo#Documentation
• https://github.com/pydap/pydap
• https://pypi.org/project/pygrib/
• https://pandas.pydata.org/
• https://xesmf.readthedocs.io/en/latest/
• https://geopandas.org/getting_started/install.html

Best regards, Sebastian

[backeb]

Hi @sebastian-luna-valero

Thanks for this, for the Aquamonitor use case openEO is a key software. I think in this ticket we could test the conda install you are working on.

cc @avgils @gena

[gena]

@gena @avgils an alternative solution to OpenEO that was put forward is: https://github.com/davidfrantz/force https://force-eo.readthedocs.io/en/latest/ @gena @avgils do you have any experience with FORCE? Nope

[Jaapel]

@jdries I am currently trying to get started doing some basic EO analysis using the OpenEO API (Python). So far nothing Big Data-ish, just want to get used to the codebase. I want to use the VITO backend instead of the GEE one, e.g.:

# imports
from openeo import connect, Connection

# Connect to VITO backend
vito_url: str = "https://openeo.vito.be/openeo/1.0"
con: Connection = connect(vito_url).authenticate_basic(username="bla", password="bla")

How can I find out how to get access? Have you guys figured out authentication yet (OIDC or basic?).

[jdries]

@Jaapel is it possible that you already managed to get access, or is this still an issue? (Just returned from vacation.)

[Jaapel]

Hi @jdries! Yes, I got help from @tcassaert and managed to connect! I heard that you were already working with @gena to create a sample script to work with Linear Regression in OpenEO for the aquamonitor use-case, is that correct?

[Jaapel]

Also just to be 100% sure, the VITO backend uses the geo-pyspark-driver as a backend, am I correct?

[jdries]

Yes indeed, that's our backend, which INCD is now also deploying. There's differences in which processes are supported by the backends, so it's good to develop your usecase on a single implementation, you can then always test portability afterwards.

I believe I provided some hints to get started with the usecase, probably using a UDF to do the linear regression, but nothing more. We can always have a call to go over possible approaches.

I'm not sure if this is useful, but new processes have been proposed in openEO to fit a curve: https://processes.openeo.org/draft/#fit_curve This is used in a change detection use case, but might be different from what aquamonitor need, and you can probably do it with UDF's anyway.

[sebastian-luna-valero]

Should this be closed?

[backeb]

Yes. This is done. See the aquamonitor notebook

---

From: Sebastian Luna-Valero @.> Sent: Tuesday, October 25, 2022 5:29:19 PM To: c-scale-community/use-case-aquamonitor @.> Cc: Björn Backeberg @.>; Comment @.> Subject: Re: [c-scale-community/use-case-aquamonitor] Reproduce a simple EE script in C-Scale using OpenEO (#3)

Caution: This message was sent from outside of Deltares. Please do not click links or open attachments unless you recognize the source of this email and know the content is safe. Please report all suspicious emails to @.***" as an attachment.

Should this be closed?

— Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fc-scale-community%2Fuse-case-aquamonitor%2Fissues%2F3%23issuecomment-1290750366&data=05%7C01%7C%7C0bf582cde52140165fbc08dab69db03d%7C15f3fe0ed7124981bc7cfe949af215bb%7C0%7C0%7C638023085634541351%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Aqnw80bij%2FxZF%2B9PFOl8NA%2BNEPra2j%2BenQgKBvz7YZY%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHAI3NQISSN6HY45UQKAY2TWE74E7ANCNFSM44ALUOQQ&data=05%7C01%7C%7C0bf582cde52140165fbc08dab69db03d%7C15f3fe0ed7124981bc7cfe949af215bb%7C0%7C0%7C638023085634541351%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=dFI%2BOl1bA6SoeYLfulKfrZ9%2BDNBYmVvdwoXpnbt%2F1%2BM%3D&reserved=0. You are receiving this because you commented.Message ID: @.***>

DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.

[sebastian-luna-valero]

Thanks!