Tests require internet access

[mbakke]

Distributions typically run tests in an isolated environment without network access. It would be great to have an option to fetch the test data from a separate directory, or optimally have them shipped with the tarball.

[gdraheim]

Well, if you're asking for a two stage option then it should actually work already. Run the test suite with some internet acess (ignore all the failures) and then save the directory "tmp.download". Unpacking it in that isolated environment should be the trick needed.

It would be a completely different thing to assemble that download-stuff within the project sources. The test data is a series of *.zip files to check with, so basically binary data. May be one could bring along that tmp.download snapshot as a secondary archive along with each release. May be.

[jtojnar]

I think I know how to fix this. Will try to do that now.

[jtojnar]

92 moves the download to CMake.

[gdraheim]

I took up @jtojnar idea to put the zziptestsdownload to cmake time.

This will allow build managers to pack the tmp.download/ space if they want to run "make check" in an environment without internet access.

As for the remaining developt, I would like to check if the download can not succeed and report the testcase as "skipped" instead of "failed". The downloads are actually zip files from CVE that have been fixed anyway, so it is not specfically needed for regressions in terms of zzip core functionality (we are going to miss out some negative testcases).

[gdraheim]

Try to disable TESTCVE to have "make check" go without downloads.

cmake -DZZIP_TESTCVE=OFF

This is also the default for the azure-piplines now.

[flokli]

@gdraheim I'd personally have preferred the solution posted by @jtojnar in https://github.com/gdraheim/zziplib/pull/92, as it provided hashes for the requested files, and fetched from a somewhat stable location.

Providing hashes for all sources, and knowing hashes for downloaded sources upfront actually is a requirement for multiple packaging tools, and the changes introduced in https://github.com/gdraheim/zziplib/commit/9758d921540b28224477708a76c99ea9a1caf909 actually just tell cmake to invoke a random script during the build process, which still requires internet access.

This effectively means packaging scripts will probably disable these tests alltogether (as you did in azure-pipelines too)

If you don't really like providing hashes, what about just moving the zipfiles to a secondary repo, and add it as a git submodule to this one?

Most packaging and CI tooling fetches submodules recursively, so the files will be just there.

[gdraheim]

@flokli - well the zip files are just binaries for tests, not a code project.

Having packagers to skip those tests is fine with me.

[flokli]

well the zip files are just binaries for tests, not a code project.

yes, which is why moving them to a separate repository would decrease the bloat. Another option would be to add them via Git LFS - it seems Azure Pipelines supports this by now.

Running these tests during the build ensures zziplib isn't vulnerable to any of the CVEs tested there - so I still think it'd be useful to be able to run these tests from inside the packaging environment.