Run tests without API keys, and without networking

Hi, I'm working on getting pyhamtools into Debian because I'm using it for my logbook [1]. I have the packaging ready [2], but the testsuite is failing because I'm running without QRZ/clublog API keys (and I don't want to give these to the Debian buildd network). Also, we have the requirements that builds shouldn't be accessing the internet. Could we maybe have

a switch to disable all the networking tests at build time, including redis?
for a separate test stage, a switch to disable all tests that require a personalized API key, so we could still test the networking bits outside the buildds? I can probably get the redis tests working at this stage

Thanks, Christoph DF7CB

[1] https://github.com/df7cb/df7cb-log [2] https://salsa.debian.org/debian-hamradio-team/pyhamtools

I: pybuild base:217: cd /srv/projects/afu/pyhamtools/pyhamtools.git/.pybuild/cpython3_3.8/build; python3.8 -m pytest test
===================================== test session starts ======================================
platform linux -- Python 3.8.1rc1, pytest-4.6.7, py-1.8.0, pluggy-0.13.0
rootdir: /srv/projects/afu/pyhamtools/pyhamtools.git
collecting ... ^Mcollecting 46 items                                                                            ^Mcollecting 249 items                                                                           ^Mcollected 275 items                                                                            

test/test_callinfo.py EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE                                   [ 11%]
test/test_clublog.py EF.                                                                 [ 12%]
test/test_dxcluster.py ...                                                               [ 13%]
test/test_eqsl.py EF.                                                                    [ 14%]
test/test_locator_distances.py .........                                                 [ 18%]
test/test_locator_latlong_to_locator.py ....                                             [ 19%]
test/test_locator_locator_to_latlong.py ......                                           [ 21%]
test/test_locator_sunrise_sunset.py ...                                                  [ 22%]
test/test_lookuplib.py ...............                                                   [ 28%]
test/test_lookuplib_clublogapi.py FF...                                                  [ 30%]
test/test_lookuplib_clublogxml.py FF.EEEEEEEEEE                                          [ 34%]
test/test_lookuplib_countryfile.py ..EEEE                                                [ 37%]
test/test_lookuplib_gettersetter_api.py .........FFFF................................... [ 54%]
.EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE [ 86%]
EEEEEEEEEEE                                                                              [ 90%]
test/test_lookuplib_qrz.py ssssssssssssss                                                [ 95%]
test/test_lookuplib_redis.py EEE                                                         [ 96%]
test/test_lotw.py EF.E                                                                   [ 98%]
test/test_utils_freq_to_band.py .....                                                    [100%]

Or maybe not new switches, but skip the tests if CLUBLOG_APIKEY and QRZ_USERNAME aren't set.

Hey Christoph,

cool - thanks for your efforts in brinding pyhamtools to Debian!

It seems that I must have tried to make the package testable without internet connection some time ago /test/conftest.py:

try:
    APIKEY = str(os.environ['CLUBLOG_APIKEY'])
    QRZ_USERNAME = str(os.environ['QRZ_USERNAME'])
    QRZ_PWD = str(os.environ['QRZ_PWD'])

except Exception as ex:
    print("WARNING: Environment variables with API keys not set; some tests will be skipped")

and /test/test_lookuplib_qrz.py

try:
    QRZ_USERNAME = str(os.environ['QRZ_USERNAME'])
    QRZ_PWD = str(os.environ['QRZ_PWD'])
except Exception:
    pytestmark = pytest.mark.skip("Environment variables with QRZ.com credentials not set")

But it seems that I haven't finished it (yet) 😢

As you pointed out, I think the best approach is to skip the clublogapi and qrz tests if the corresponding variables CLUBLOG_APIKEY, QRZ_USERNAME and QRZ_PWD are not set. However I think that there are a few more tests e.g. for countryfiles where the lookup files can be downloaded publicly without an API key. So I think we can not entirely rely on checking if these variables are set.

That said, for some of the tests I'm using static fixtures in order to ensure reproducibility of the tests. All of those fixtures are located in /test/fixtures folder. Some of them are used directly as fixtures, others are provided through a mocked http server, for example in /test/test_lotw.py.

WRT Redis I'm not sure what is the best way. Maybe a brief check if the redis client can connect to the redis server on port 6379 during setup in /test/conftest.py

I think I will deprecate redis anyway in the long term. There are now so may cache solutions out there available. I think it should be therefore left to the user to decide which cache system to use... but that's for another day 😉

I'm kinda busy the next 2 weeks and I can't promise that I'll be able to include all the changes this year. But if you are in the mood, I would happily accept a PR!

73 Tobias DH1TW

Hi Tobias, thanks for your efforts. In the meantime, the package was accepted into Debian/unstable.

For the test, I'm not very fluent in python yet, and we don't really need to run the full testsuite in Debian, so I put a limited smoke test that simply calls a few functions into https://salsa.debian.org/debian-hamradio-team/pyhamtools/blob/debian/debian/tests/test. But if you get the testsuite running without the API keys, I'll definitely revisit that and try to run it from the Debian CI pipeline.

73, Christoph

dh1tw / pyhamtools

Run tests without API keys, and without networking #13