stactools-packages / esa-cci-lc

stactools package for ESA's Climate Change Initiative (CCI) Land Cover (LC) product
Other
6 stars 0 forks source link

Add tests for Items through external test data #2

Closed m-mohr closed 1 year ago

m-mohr commented 2 years ago

Test against two online files

Run tests without COGs for push/PR and with COGs for releases.

gadomski commented 2 years ago

I've added two external data urls in this branch: https://github.com/stactools-packages/esa-cci-lc/tree/add-external-data. However, when I tried to download, it eventually timed out. We may have to have manual external data instructions instead rather than an automatic process, since these files as so big.

Alternatively, we could try to subset the netcdfs?

m-mohr commented 2 years ago

Hmm, or extend the timeout? I assume in CI it would actually be faster because we are not limited by slow home internet connections.

Manual external instructions would essentially mean no CI for tests, which is unfortunate.

Unfortunately, I did not manage to subset the files yet.

I also faced some questions: https://github.com/stac-utils/stactools/issues/364

By the way, I always worked on the pure nc files, not on archive files. This is due to the fact that you can download multiple files either as zip or tar.gz from CDS, so I'd think extraction needs to happen beforehand to be explicit about which file(s) to convert.

m-mohr commented 2 years ago

I've merged the tests to the main branch. I've added instructions how to download the tests files and where to place them. If files are missing the tests are skipped. Unfortunately, I did not have the time to make the external download for tests work.

Details: https://github.com/stactools-packages/esa-cci-lc#tests

m-mohr commented 2 years ago

Further implementing external test data handling through stactools in the add-external-data branch did not work due to the following upstream issues:

pjhartzell commented 1 year ago

Closed by #11