[JOSS] GEE: onboarding issues

[ashwinvis]

Possibly typos / mistakes in the docs

My comments in italics.

https://vergauwenthomas.github.io/MetObs_toolkit/topics/gee_authentication.html#setup-a-cloud-project-on-your-developer-account

• The API’s that are used by the toolkit has quite a lot of free credentials

Free credits?

Also are you implying that we need to rely on the $300 USD free credits that new users get, or is it that the API itself is free, or is it that the toolkit supplies the necessary credentials to use this API? I am confused.

https://vergauwenthomas.github.io/MetObs_toolkit/topics/gee_authentication.html#enable-api-s-on-your-project

• Search for the ‘Google Earth Engine API’, click on it –> hit ENABLE

There is no such ENABLE button to click. It says API ENABLED, so perhaps they automatically enabled it?

https://vergauwenthomas.github.io/MetObs_toolkit/topics/gee_authentication.html#test-your-gee-access

_This seems to imply that a dependency was missing from the installation instructions, but the only thing I had to do was to upgrade pip install earthengine-api --upgrade__

>>> dataset.get_lcz()
Fetching credentials using gcloud
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
File ~/test-metobs/lib/python3.11/site-packages/ee/oauth.py:317, in _load_app_default_credentials(run_gcloud, scopes, quiet)
    316 try:
--> 317   subprocess.run(command, check=True)
    318 except FileNotFoundError as e:

...
File ~/test-metobs/lib/python3.11/site-packages/ee/oauth.py:320, in _load_app_default_credentials(run_gcloud, scopes, quiet)
    318 except FileNotFoundError as e:
    319   tip = 'Please ensure that gcloud is installed.' + more_info
--> 320   raise Exception('gcloud command not found. ' + tip) from e  # pylint:disable=broad-exception-raised
    321 except subprocess.CalledProcessError as e:
    322   tip = ('Please check for any errors above.\n*Possible fixes:'
    323          ' If you loaded a page with a "redirect_uri_mismatch" error,'
    324          ' run earthengine authenticate with the --quiet flag;'
    325          ' if the error page says "invalid_request", be sure to run the'
    326          ' entire gcloud auth command that is shown.' + more_info)

Exception: gcloud command not found. Please ensure that gcloud is installed.
More information: https://developers.google.com/earth-engine/guides/python_install

And then upon upgrading, and following the authentication link

~/test-metobs/lib/python3.11/site-packages/metobs_toolkit/dataset.py:2886, in Dataset.get_lcz(self)
   2872 """Extract local climate zones for all stations.
   2873
   2874 Function to extract the Local CLimate zones (LCZ) from the
   (...)
   2883
   2884 """
   2885 # connect to gee
-> 2886 connect_to_gee()
   2888 # Extract LCZ for all stations
   2889 lcz_series = lcz_extractor(
   2890     metadf=self.metadf,
   2891     mapinfo=self.settings.gee["gee_dataset_info"]["global_lcz_map"],
   2892 )

...

File ~/test-metobs/lib/python3.11/site-packages/ee/data.py:404, in _execute_cloud_call(call, num_retries)
    402   return call.execute(num_retries=num_retries)
    403 except googleapiclient.errors.HttpError as e:
--> 404   raise _translate_cloud_exception(e)

EEException: Not signed up for Earth Engine or project is not registered. Visit https://developers.google.com/earth-engine/guides/access

Right now I am stuck here. How do I fix this?

[ashwinvis]

I tried to move ~/.config/earthengine/credentials file to redo the process but I end up with the same EEException again. Strange, since I can see a project with the API enabled:

[vergauwenthomas]

@ashwinvis i will take a look into it. Could you test if this results in the same error:

import ee

ee.Authenticate()
ee.Initialize()

[ashwinvis]

EEException: Not signed up for Earth Engine or project is not registered. Visit https://developers.google.com/earth-engine/guides/access

Same issue as before :tired_face:

[vergauwenthomas]

Did you manage to enable the google earth engine (red arrow) and then you need to register it (green arrow) ?

[vergauwenthomas]

Could you click on this link: code.earthengine.google.com/client-auth

This link should bring you to an authentication window, where you can select the project to use the API for.

[ashwinvis]

Did you manage to enable the google earth engine (red arrow)

Not entirely sure, but I guess so.

and then you need to register it (green arrow) ?

Yes

[ashwinvis]

Could you click on this link: code.earthengine.google.com/client-auth

This link should bring you to an authentication window, where you can select the project to use the API for.

I can see it, but if I click on GENERATE TOKEN, I get "Invalid request"

[vergauwenthomas]

Hi @ashwinvis, i made a short instruction video on how to set up the project. You can find it here: https://youtu.be/h_8kX_oJulw

To test the API and your project, i used this authentication method:

import ee
ee.Authenticate(force=True) #force to create new credentials
ee.Initialize()

Can you reproduce these steps and let me know if this works, or where is does not work?

[ashwinvis]

I see what went wrong. I was always trying from IPython console, and then authentication process is very different from the Notebook Authenticator. If I try that method, the ~/.config/earthengine/credentials which gets saved, contains no client_id and client_secret fields.

Now that I tried it via JupyterLab a correct credentials file is saved.

[ashwinvis]

I think you either try to do exception handling and / or note this known issue in the documentation. Recommend to use Jupyter instead.

[vergauwenthomas]

@ashwinvis, This issue is addressed in #444 where

• The user can now specify which authentication method is used. (Which solves the invalid credential file + the gcloud dependency)
• The authentication function is callable for the user (metobs_toolkit.connect_to_gee()) with kwargs passed to the ee.Authenticate().
• This issue, and proposed solution is addressed in the documentation in the Using google earth engine page as a warning banner (see screenshot)
• Typo/explenation on free credits is fixed in the documentation.

@ashwinvis let me know if you agree that #444 solves this issue, so i can merge.

[vergauwenthomas]

Ref: https://github.com/openjournals/joss-reviews/issues/5916

[ashwinvis]

LGTM! Go ahead :+1: