search

initze / thaw-slump-segmentation

MIT License
10 stars 4 forks source link

gcloud preprocessing bug #110

Closed initze closed 5 months ago

initze commented 5 months ago

Earthengine Initialization fails in current version for setup_raw_data.py

Preprocessing 17 images [2024-04-26T14:02:13.237+02:00] uc2.setup_raw_data INFO: ########################### [2024-04-26T14:02:13.237+02:00] uc2.setup_raw_data INFO: # Starting Raw Data Setup # [2024-04-26T14:02:13.237+02:00] uc2.setup_raw_data INFO: ########################### [2024-04-26T14:02:16.794+02:00] uc2.setup_raw_data.5787151_4270616_2022-07-15_2427 WARNING: Initializing Earth Engine failed, trying to authenticate [2024-04-26T14:02:16.801+02:00] uc2.setup_raw_data.4709837_4271320_2021-07-20_2450 WARNING: Initializing Earth Engine failed, trying to authenticate [2024-04-26T14:02:16.811+02:00] uc2.setup_raw_data.5790504_4270718_2022-07-15_247e WARNING: Initializing Earth Engine failed, trying to authenticate [2024-04-26T14:02:16.849+02:00] uc2.setup_raw_data.5828843_4171215_2022-08-02_2424 WARNING: Initializing Earth Engine failed, trying to authenticate [2024-04-26T14:02:16.869+02:00] uc2.setup_raw_data.5784931_3972918_2022-07-14_2421 WARNING: Initializing Earth Engine failed, trying to authenticate [2024-04-26T14:02:16.896+02:00] uc2.setup_raw_data.5804928_5472217_2022-07-23_2431 WARNING: Initializing Earth Engine failed, trying to authenticate [2024-04-26T14:02:16.901+02:00] uc2.setup_raw_data.5790504_4270617_2022-07-15_247e WARNING: Initializing Earth Engine failed, trying to authenticate [2024-04-26T14:02:16.903+02:00] uc2.setup_raw_data.4884586_4371817_2021-09-10_1010 WARNING: Initializing Earth Engine failed, trying to authenticate [2024-04-26T14:02:16.904+02:00] uc2.setup_raw_data.5794680_4270916_2022-07-18_2464 WARNING: Initializing Earth Engine failed, trying to authenticate [2024-04-26T14:02:16.904+02:00] uc2.setup_raw_data.5817039_5270718_2022-07-28_2414 WARNING: Initializing Earth Engine failed, trying to authenticate [2024-04-26T14:02:16.910+02:00] uc2.setup_raw_data.5856953_4873412_2022-08-15_2458 WARNING: Initializing Earth Engine failed, trying to authenticate Fetching credentials using gcloud [2024-04-26T14:02:16.915+02:00] uc2.setup_raw_data.5808346_4572815_2022-07-24_247b WARNING: Initializing Earth Engine failed, trying to authenticate Fetching credentials using gcloud [2024-04-26T14:02:16.933+02:00] uc2.setup_raw_data.5794685_4270514_2022-07-18_242d WARNING: Initializing Earth Engine failed, trying to authenticate [2024-04-26T14:02:16.936+02:00] uc2.setup_raw_data.5797073_4573012_2022-07-19_241f WARNING: Initializing Earth Engine failed, trying to authenticate Fetching credentials using gcloud [2024-04-26T14:02:16.945+02:00] uc2.setup_raw_data.4709837_4371511_2021-07-20_2450 WARNING: Initializing Earth Engine failed, trying to authenticate Fetching credentials using gcloud [2024-04-26T14:02:16.965+02:00] uc2.setup_raw_data.5792032_4271118_2022-07-17_2475 WARNING: Initializing Earth Engine failed, trying to authenticate Fetching credentials using gcloud Fetching credentials using gcloud Fetching credentials using gcloud Fetching credentials using gcloud Fetching credentials using gcloud Fetching credentials using gcloud [2024-04-26T14:02:17.036+02:00] uc2.setup_raw_data.4710590_4572611_2021-07-20_2212 WARNING: Initializing Earth Engine failed, trying to authenticate Fetching credentials using gcloud Fetching credentials using gcloud Fetching credentials using gcloud Fetching credentials using gcloud Fetching credentials using gcloud Fetching credentials using gcloud Fetching credentials using gcloud joblib.externals.loky.process_executor._RemoteTraceback: """ Traceback (most recent call last): File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/thaw_slump_segmentation/scripts/setup_raw_data.py", line 49, in preprocess_directory ee.Initialize() File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/ee/_utils.py", line 38, in wrapper return func(*args, kwargs) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/ee/init.py", line 160, in Initialize data.initialize( File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/ee/data.py", line 225, in initialize _install_cloud_api_resource() File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/ee/data.py", line 319, in _install_cloud_api_resource _cloud_api_resource = _cloud_api_utils.build_cloud_resource( File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/ee/_cloud_api_utils.py", line 207, in build_cloud_resource resource = build(static_discovery=False) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/ee/_cloud_api_utils.py", line 191, in build return discovery.build( File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper return wrapped(*args, *kwargs) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/googleapiclient/discovery.py", line 304, in build content = _retrieve_discovery_doc( File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/googleapiclient/discovery.py", line 439, in _retrieve_discovery_doc resp, content = req.execute(num_retries=num_retries) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper return wrapped(args, kwargs) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/googleapiclient/http.py", line 923, in execute resp, content = _retry_request( File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/googleapiclient/http.py", line 191, in _retry_request resp, content = http.request(uri, method, *args, **kwargs) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/google_auth_httplib2.py", line 209, in request self.credentials.before_request(self._request, method, uri, request_headers) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/google/auth/credentials.py", line 230, in before_request self._blocking_refresh(request) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/google/auth/credentials.py", line 193, in _blocking_refresh self.refresh(request) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/google/oauth2/credentials.py", line 431, in refresh ) = reauth.refresh_grant( File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/google/oauth2/reauth.py", line 365, in refresh_grant _client._handle_error_response(response_data, retryable_error) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/google/oauth2/_client.py", line 72, in _handle_error_response raise exceptions.RefreshError( google.auth.exceptions.RefreshError: ('invalid_scope: Bad Request', {'error': 'invalid_scope', 'error_description': 'Bad Request'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/ee/oauth.py", line 371, in _load_gcloud_credentials subprocess.run(command, check=True) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/subprocess.py", line 503, in run with Popen(*popenargs, **kwargs) as process: File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/subprocess.py", line 971, in init self._execute_child(args, executable, preexec_fn, close_fds, File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/subprocess.py", line 1863, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: 'gcloud'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/joblib/externals/loky/process_executor.py", line 463, in _process_worker r = call_item() File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/joblib/externals/loky/process_executor.py", line 291, in call return self.fn(*self.args, self.kwargs) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/joblib/parallel.py", line 588, in call return [func(*args, *kwargs) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/joblib/parallel.py", line 588, in return [func(args, kwargs) File "/isipd/projects-noreplica/p_initze/anaconda3/envs/RTS_v010/lib/python3.10/site-packages/thaw_slump_segmentation/scripts/setup_raw_data.py", line 52, in preprocess_directory ee.Authenticate()

initze commented 5 months ago

reqeuirements for earthengine to older versions