Closed jdries closed 8 months ago
To write a test for this, we need to mock a call that returns a lot of products: https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?box=-1.3258149867294418%2C52.34731665664588%2C1.6910298219426412%2C54.32658696166628&sortParam=startDate&sortOrder=ascending&page=1&maxRecords=2000&status=ONLINE&dataset=ESA-DATASET&productType=L2A&cloudCover=[0%2C75]&startDate=2022-01-01T00%3A00%3A00Z&completionDate=2022-12-31T00%3A00%3A00Z
Then we need to perform sparse sampling, and see if we can construct a datacube within a reasonable time.
in actual job, the RDD used 12GB so that would need to be reduced
Fixed on staging! Memory use got reduced quite a lot for sparse sampling case, dense sampling might still require a bit more.
To write a test for this, we need to mock a call that returns a lot of products: https://catalogue.dataspace.copernicus.eu/resto/api/collections/Sentinel2/search.json?box=-1.3258149867294418%2C52.34731665664588%2C1.6910298219426412%2C54.32658696166628&sortParam=startDate&sortOrder=ascending&page=1&maxRecords=2000&status=ONLINE&dataset=ESA-DATASET&productType=L2A&cloudCover=[0%2C75]&startDate=2022-01-01T00%3A00%3A00Z&completionDate=2022-12-31T00%3A00%3A00Z
Then we need to perform sparse sampling, and see if we can construct a datacube within a reasonable time.
in actual job, the RDD used 12GB so that would need to be reduced