Skip tests if internet is not available

javierggt commented 5 months ago

Description

Skip tests if internet is not available.

Interface impacts

None

Testing

Unit tests

[x] Mac (with WiFi on and off by @taldcroft)

masters) ➜  kadi git:(skip-test) git rev-parse HEAD
b47f543e7f61acd180fda7b649e0358c8411f39b

Without internet (wifi off)


(masters) ➜  kadi git:(skip-test) pytest
===================================================== test session starts =====================================================
platform darwin -- Python 3.10.8, pytest-7.2.1, pluggy-1.0.0
rootdir: /Users/aldcroft/git, configfile: pytest.ini
plugins: timeout-2.1.0, anyio-3.6.2
collected 174 items

kadi/commands/tests/test_commands.py ........sssss..ss...............................................ss...s...... [ 43%] kadi/commands/tests/test_states.py ......................x....................... [ 70%] kadi/commands/tests/test_validate.py .sssssssssssssssssss [ 81%] kadi/tests/test_events.py .......... [ 87%] kadi/tests/test_occweb.py ssssssssssssssssssssss [100%]

========================================= 122 passed, 51 skipped, 1 xfailed in 16.44s =========================================

### With internet

(masters) ➜ kadi git:(skip-test) pytest ===================================================== test session starts ===================================================== platform darwin -- Python 3.10.8, pytest-7.2.1, pluggy-1.0.0 rootdir: /Users/aldcroft/git, configfile: pytest.ini plugins: timeout-2.1.0, anyio-3.6.2 collected 228 items

kadi/commands/tests/test_commands.py .................................................................................. [ 35%] .. [ 36%] kadi/commands/tests/test_states.py ......................x.............................................x............... [ 73%] ........ [ 77%] kadi/commands/tests/test_validate.py .................... [ 85%] kadi/tests/test_events.py .......... [ 90%] kadi/tests/test_occweb.py ...................... [100%]

========================================= 226 passed, 2 xfailed in 100.15s (0:01:40) ==========================================



### Functional tests
<!-- Describe and document results of any functional tests, otherwise leave the text below -->
No functional testing.

taldcroft commented 5 months ago

@javierggt - I tested this by turning off my wifi and found some other failures. In addition, the validation tests should succeed if there is internet. If the command sheet disappeared for some reason then we definitely want tests to fail.

javierggt commented 5 months ago

I suppose turning off wifi introduces extra failures. I checked that this PR skipped the failing tests, so tests pass on GRETA.

If we want to have tests fail if the command sheet is not reachable, then I suppose the only solution is to explicitly skip on GRETA, but I am assuming that having a list of GRETA machines is not what you want. Do we have a test that would tell us we are on GRETA?

taldcroft commented 5 months ago

If we want to have tests fail if the command sheet is not reachable, then I suppose the only solution is to explicitly skip on GRETA, but I am assuming that having a list of GRETA machines is not what you want. Do we have a test that would tell us we are on GRETA?

has_internet() checks that https://google.com is reachable. If that is true then all the command states functionality is expected to work. All the commands V2 tests (which depend on the same Google Sheet) use has_internet() as the skipif test, so applying this to validation makes sense to me. If the machine can reach google.com but not the Command Sheet, then we have a big problem.

The only risk in this strategy would be if HEAD was failing has_internet() since that would mean silently skipping tests that should really fail. But this is not a credible situation at least for any extended time.

We don't have a test that says we are on GRETA, the goal in unit tests is to ask whether required resources are available for a particular test. We do have a test for being on HEAD which gets used in a few places where this helps (https://github.com/search?q=org%3Asot%20on_head_network&type=code).

sot / kadi