Closed m-mohr closed 1 year ago
I've added two external data urls in this branch: https://github.com/stactools-packages/esa-cci-lc/tree/add-external-data. However, when I tried to download, it eventually timed out. We may have to have manual external data instructions instead rather than an automatic process, since these files as so big.
Alternatively, we could try to subset the netcdfs?
Hmm, or extend the timeout? I assume in CI it would actually be faster because we are not limited by slow home internet connections.
Manual external instructions would essentially mean no CI for tests, which is unfortunate.
Unfortunately, I did not manage to subset the files yet.
I also faced some questions: https://github.com/stac-utils/stactools/issues/364
By the way, I always worked on the pure nc files, not on archive files. This is due to the fact that you can download multiple files either as zip or tar.gz from CDS, so I'd think extraction needs to happen beforehand to be explicit about which file(s) to convert.
I've merged the tests to the main branch. I've added instructions how to download the tests files and where to place them. If files are missing the tests are skipped. Unfortunately, I did not have the time to make the external download for tests work.
Details: https://github.com/stactools-packages/esa-cci-lc#tests
Further implementing external test data handling through stactools in the add-external-data branch did not work due to the following upstream issues:
Closed by #11
Test against two online files
Run tests without COGs for push/PR and with COGs for releases.