search

opendatacube / odc-stats

Statistician is a framework of tools for generating statistical summaries of large collections of EO data managed in an ODC instance.
Apache License 2.0
9 stars 4 forks source link

Update the docker image to use the latest odc-stats version 1.0.33 #87

Closed vikineema closed 1 year ago

vikineema commented 1 year ago

The docker image tagged latest uses the odc-stats version 1.0.32 instead of the latest odc-stats version which is 1.0.33. docker container run -it opendatacube/datacube-statistician:latest pip list| grep odc-stats

odc-stats 1.0.32

fangfy commented 1 year ago

Hi @emmaai @SpacemanPaul @omad, it looks like dockerise test failed (https://github.com/opendatacube/odc-stats/actions/runs/4191291551/jobs/7265577644). Any idea why?

emmaai commented 1 year ago

The connection is reset by s3. Rerun will fix it.

vikineema commented 1 year ago

Hi @emmaai @Spac @omad . I've tested the latest published image from the rerun workflow and the odc-stats version is still 1.0.32. What could be causing this?

emmaai commented 1 year ago

had a look, it used the cache available in ecr, since nothing has changed with the files involved in the docker image building.

fangfy commented 1 year ago

Hi @emmaai, do you have an estimate of when this can be fixed? we urgently need this new image to generate S2 gm products.

jmettes commented 1 year ago

I've been noticing a lot of intermittent failures in tests using external S3 data in odc-tools - especially those using s3://sentinel-s2-l2a-cogs. I wonder if that's what's happening here too. I wonder if that bucket recently changed to a less highly-available storage class. Ideally these tests would be replaced with mocks, or locally stored data.

Maybe in the meantime, it's worth trying to set GDAL_HTTP_MAX_RETRY somewhere? https://github.com/rasterio/rasterio/issues/2119#issuecomment-790024225

Apparently max_retry default is 0: https://gdal.org/user/virtual_file_systems.html

emmaai commented 1 year ago

Hi @emmaai, do you have an estimate of when this can be fixed? we urgently need this new image to generate S2 gm products.

Within this week.

emmaai commented 1 year ago

I've been noticing a lot of intermittent failures in tests using external S3 data in odc-tools - especially those using s3://sentinel-s2-l2a-cogs. I wonder if that's what's happening here too. I wonder if that bucket recently changed to a less highly-available storage class. Ideally these tests would be replaced with mocks, or locally stored data.

Maybe in the meantime, it's worth trying to set GDAL_HTTP_MAX_RETRY somewhere? rasterio/rasterio#2119 (comment)

Apparently max_retry default is 0: https://gdal.org/user/virtual_file_systems.html

For the test, GDAL_HTTP_MAX_RETRY is set here https://github.com/opendatacube/odc-stats/blob/0826115ffc577fc46d9d80b11fa1ae872c3cf61a/docker/docker-compose.yml#L21 and here https://github.com/opendatacube/odc-stats/blob/0826115ffc577fc46d9d80b11fa1ae872c3cf61a/.github/workflows/main.yml#L137

Still, the connection can be reset by s3 server, which can not be caught or retried.

emmaai commented 1 year ago

Could you have a look at PR #88, it should resolve the issue.