Servir-Mekong / hydra-floods

HYDrologic Remote sensing Analysis for Floods Python package
https://servir-mekong.github.io/hydra-floods/
GNU General Public License v3.0
162 stars 48 forks source link

Error in dswfp.export_fusion_samples() #15

Closed s-boeck closed 3 years ago

s-boeck commented 3 years ago

Running

# define a geographic region and time period
region = hf.country_bbox("South Sudan")
start_time = "2019-01-01"
end_time = "2019-03-01"

# define the asset name of the output
output_asset = (
    "users/MYNAME/fusion_sampling_"+
    f"{start_time.replace('-','')}_{end_time.replace('-','')}"
)

# run the sampling process
dswfp.export_fusion_samples(
    region,
    start_time,
    end_time,
    stratify_samples=True,
    sample_scale=100,
    output_asset_path=output_asset
)

I receive this ERROR message

---------------------------------------------------------------------------
HttpError                                 Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/ee/data.py in _execute_cloud_call(call, num_retries)
    344   try:
--> 345     return call.execute(num_retries=num_retries)
    346   except googleapiclient.errors.HttpError as e:

10 frames
HttpError: <HttpError 400 when requesting https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/value:compute?prettyPrint=false&alt=json returned "Collection.reduceColumns: Error in map(ID=20190101T125309_20190101T082935_T35PRP):
Image.select: Parameter 'input' is required.">

During handling of the above exception, another exception occurred:

EEException                               Traceback (most recent call last)
<decorator-gen-60> in time(self, line, cell, local_ns)

<timed eval> in <module>()

/usr/local/lib/python3.6/dist-packages/ee/data.py in _execute_cloud_call(call, num_retries)
    345     return call.execute(num_retries=num_retries)
    346   except googleapiclient.errors.HttpError as e:
--> 347     raise _translate_cloud_exception(e)
    348 
    349 

EEException: Collection.reduceColumns: Error in map(ID=20190101T125309_20190101T082935_T35PRP):
Image.select: Parameter 'input' is required.

Doesn't complain when changing country_bbox to Cambodia

Best regards, Sebastian

ArjenHaag commented 3 years ago

Seems to be within the LSIB dataset that is used, i.e. ee.FeatureCollection("USDOS/LSIB_SIMPLE/2017"). If I try to filter by name on 'South Sudan' it does not return anything, even though it is really listed as such (see example below where I can find it by filtering on object ID, but only knew which ID it was after using inspector on complete dataset):

https://code.earthengine.google.com/93a12d66d9f1433cb574003b2f341cd7

Should probably notify the GEE team and/or LSIB data provider?

Regardless, it might also be interesting to switch to the FAO GAUL dataset? This seems to be a bit more accurate (e.g. for Cambodia, it also includes some of the islands that are not included with LSIB) and also has multiple administrative levels (e.g. provinces, districts).

KMarkert commented 3 years ago

As @ArjenHaag mentioned this is an issue with the underlying EE dataset that the function relies on. The hf.country_bbox("South Sudan") is a helper function to get an ee.Geometry bounding box around the country. To replicate this functionality for South Sudan, you can simply define the region using the EE API:

region = ee.Geometry.Rectangle([23.4371, 3.4862, 35.9486, 12.2381])