c-scale-community / use-case-waterwatch

Apache License 2.0
0 stars 0 forks source link

GCP dependency in the openEO notebook? #14

Closed sebastian-luna-valero closed 1 year ago

sebastian-luna-valero commented 1 year ago

Hi,

When running:

https://github.com/c-scale-community/use-case-waterwatch/blob/main/notebooks/openeo_waterwatch.ipynb

There is the cell

# Get reservoirs from database
reservoir_dir: Path = out_dir / "reservoirs"

reservoirs: List[Reservoir] = Reservoir.from_gcp(reservoir_dir)

Is this expected?

Context: I would like to configure this repo for binder: xref. Then make a new release so it's ingested via Zenodo.

Jaapel commented 1 year ago

Hi Sebastian, I cannot even start the binder notebook:

Err:5 http://packages.cloud.google.com/apt cloud-sdk InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B53DC80D13EDEF05
Reading package lists...
W: GPG error: http://packages.cloud.google.com/apt cloud-sdk InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY B53DC80D13EDEF05
E: The repository 'http://packages.cloud.google.com/apt cloud-sdk InRelease' is not signed.
Removing intermediate container f9c15d2155d3
The command '/bin/bash -o pipefail -c apt-get update -y && apt-get install -y     apt-transport-https     ca-certificates     curl     gnupg  && echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] http://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list  && curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | tee /usr/share/keyrings/cloud.google.gpg  && apt-get update -y  && apt-get install google-cloud-sdk -y  && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100

Did you also encounter this issue? I can build the docker image locally, also on your branch

sebastian-luna-valero commented 1 year ago

Hi Jaap,

Please try this instead:

https://replay.notebooks.egi.eu/v2/gh/c-scale-community/use-case-waterwatch/binder?labpath=notebooks%2Fopeneo_waterwatch.ipynb

Jaapel commented 1 year ago

I pushed a commit to fix this authentication issue, can you retry?

sebastian-luna-valero commented 1 year ago

I get now:

File /srv/conda/envs/notebook/lib/python3.9/site-packages/google/auth/credentials.py:223, in AnonymousCredentials.refresh(self, request)
    220 def refresh(self, request):
    221     """Raises :class:``InvalidOperation``, anonymous credentials cannot be
    222     refreshed."""
--> 223     raise exceptions.InvalidOperation("Anonymous credentials cannot be refreshed.")

InvalidOperation: Anonymous credentials cannot be refreshed.
Jaapel commented 1 year ago

I cleaned and tested it myself now, it should work now

sebastian-luna-valero commented 1 year ago

Sorry, I am getting now:

FileNotFoundError: [Errno 2] No such file or directory: 'output/reservoirs/reservoirs-v1.0.dbf'
Jaapel commented 1 year ago

I created the out_dir now by default,

sebastian-luna-valero commented 1 year ago

Thanks!

I get this warning:

image

And this error:

image

Jaapel commented 1 year ago

Thanks! Seems like some versions changed at your end, let's see where this changed. 1st warning seems an update to the collection of the global surface water dataset, also on my TODOs now.

sebastian-luna-valero commented 1 year ago

Thanks!

I used conda-lock to create a permanent environment: https://github.com/c-scale-community/use-case-waterwatch/blob/binder/conda-linux-64.lock

I was planning to make the release with this permanent environment.

Jaapel commented 1 year ago

I fixed openEO to 0.22 now

sebastian-luna-valero commented 1 year ago

New report ;)

See error:

image

Warning:

image

Success?

image

Jaapel commented 1 year ago

About the proj error: https://github.com/Toblerity/Fiona/issues/849, seems something went wrong with conda + binder, that PROJ_LIB is not set correctly. I do not have this error locally. The warning will stay, that is for the backend guys to solve, as I am not renaming any of these bands. Success indeed if you get the timeseries + watermask images!

sebastian-luna-valero commented 1 year ago

I guess this one is the watermask images?

image

sebastian-luna-valero commented 1 year ago

Are you happy with this?

Should we merge and make a release?

Jaapel commented 1 year ago

Yes it is looking good! You can merge if you want!

sebastian-luna-valero commented 1 year ago

great, will do, thanks!