search

halfgaar / FlashMQ

FlashMQ is a fast light-weight MQTT broker/server, designed to take good advantage of multi-CPU environments
https://www.flashmq.org/
Open Software License 3.0
173 stars 24 forks source link

Feature request: Allow skipping tests that require a network connection #105

Closed quinox closed 1 month ago

quinox commented 1 month ago

I install FlashMQ using Portage, my package manager. I wanted to enable running the tests before replacing my system's FlashMQ but unfortunately this is currently not possible:

Tests run: 133. Passed: 128. Failed: 5 (of which 0 exceptions). Total assertions: 356423.

Failed tests:
 - testAsyncCurl
 - testDnsResolver
 - testDnsResolverDontCancel
 - testDnsResolverInvalid
 - testDnsResolverSecondQuery

Gentoo runs testcases in a sandbox and won't allow network connections, making it impossible to pass all the testcases.

They documented their reasons for this (link):

  • the build may be running in an environment with no or restricted Internet access, and this must not cause the tests (build) to fail;
  • the Internet connection may be unstable (e.g. poor reception) in which case an interrupted connection or packet loss must not cause the tests to fail or hang, and it should not cause unnecessary delays;
  • the Internet connection may be running on a limited data plan in which case the additional network use may cause additional charges or other inconveniences to the user;
  • the remote network services used by the tests may become unavailable temporarily or permanently, causing unexpected test failures;
  • accessing remote sites always poses a privacy issue, and possibly a threat to security (e.g. through inadvertently exposing information about the system).

It would be nice if the FlashMQ testing framework has an easy way to skip the tests that require a network connection. Gentoo says the best solution would be to use mocking / replay data in the testcases but that sounds annoying to set up and maintain.

Attached my ebuild (just FYI, it's not really relevant to the feature request): flashmq-1.13.0.ebuild.txt

halfgaar commented 1 month ago

I can do this. I'll do it along with making groups, because there are also many tests that don't require FlashMQ to run.

halfgaar commented 1 month ago

Fixed in master. Your ebuild refers to tags, so it won't work until the next release. Not sure when that is.