search

kbandla / dpkt

fast, simple packet creation / parsing, with definitions for the basic TCP/IP protocols
Other
1.09k stars 270 forks source link

1.9.5: pyest is failing when `pytest-randomly` is used #564

Closed kloczek closed 3 years ago

kloczek commented 3 years ago

It would be goot if dpkt would be pytest ready as tox has a bit bigger dependencies than pytes

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-dpkt-1.9.5-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-dpkt-1.9.5-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/python3 -Bm pytest -ra -k '  not test_deprecated_decorator'
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.9, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/tkloczko/rpmbuild/BUILD/dpkt-1.9.5/.hypothesis/examples')
rootdir: /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.5, configfile: setup.cfg
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, asyncio-0.14.0, expect-1.1.0, cov-2.11.1, mock-3.5.1, httpbin-1.0.0, xdist-2.2.1, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, pyfakefs-4.4.0, freezegun-0.4.2, flaky-3.7.0, cases-3.4.6, hypothesis-6.10.0
collected 417 items / 3 errors / 414 selected

================================================================================== ERRORS ==================================================================================
________________________________________________________________________ ERROR collecting setup.py _________________________________________________________________________
/usr/lib64/python3.8/distutils/fancy_getopt.py:233: in getopt
    opts, args = getopt.getopt(args, short_opts, self.long_opts)
/usr/lib64/python3.8/getopt.py:95: in getopt
    opts, args = do_shorts(opts, args[0][1:], shortopts, args[1:])
/usr/lib64/python3.8/getopt.py:195: in do_shorts
    if short_has_arg(opt, shortopts):
/usr/lib64/python3.8/getopt.py:211: in short_has_arg
    raise GetoptError(_('option -%s not recognized') % opt, opt)
E   getopt.GetoptError: option -r not recognized

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/distutils/core.py:134: in setup
    ok = dist.parse_command_line()
/usr/lib64/python3.8/distutils/dist.py:475: in parse_command_line
    args = parser.getopt(args=self.script_args, object=self)
/usr/lib64/python3.8/distutils/fancy_getopt.py:235: in getopt
    raise DistutilsArgError(msg)
E   distutils.errors.DistutilsArgError: option -r not recognized

During handling of the above exception, another exception occurred:
setup.py:24: in <module>
    setup(name=package_name,
/usr/lib/python3.8/site-packages/setuptools/__init__.py:144: in setup
    return distutils.core.setup(**attrs)
/usr/lib64/python3.8/distutils/core.py:136: in setup
    raise SystemExit(gen_usage(dist.script_name) + "\nerror: %s" % msg)
E   SystemExit: usage: __main__.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
E      or: __main__.py --help [cmd1 cmd2 ...]
E      or: __main__.py --help-commands
E      or: __main__.py cmd --help
E
E   error: option -r not recognized
______________________________________________________________________ ERROR collecting docs/conf.py _______________________________________________________________________
docs/conf.py:48: in <module>
    release = get_version('../dpkt/__init__.py')
docs/conf.py:14: in get_version
    init_py = open(filename).read()
E   FileNotFoundError: [Errno 2] No such file or directory: '../dpkt/__init__.py'
_______________________________________________________________ ERROR collecting examples/old/dhcprequest.py _______________________________________________________________
examples/old/dhcprequest.py:24: in <module>
    interface = pcapy.open_live(iface_name, 65536, 1, 0)
E   pcapy.PcapError: eno0: You don't have permission to capture on that device (socket: Operation not permitted)
----------------------------------------------------------------------------- Captured stdout ------------------------------------------------------------------------------
Auto Setting Interface to: eno0
========================================================================= short test summary info ==========================================================================
ERROR setup.py - SystemExit: usage: __main__.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
ERROR docs/conf.py - FileNotFoundError: [Errno 2] No such file or directory: '../dpkt/__init__.py'
ERROR examples/old/dhcprequest.py - pcapy.PcapError: eno0: You don't have permission to capture on that device (socket: Operation not permitted)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 3 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================ 3 errors in 2.53s =============================================================================
brifordwylie commented 3 years ago

All of the automated tests use pytest... in the root directory of the repository.. you simply run pytest dpkt

> pytest dpkt
============================= test session starts ==============================
platform darwin -- Python 3.9.1, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /Users/briford/.pyenv/versions/3.9.1/envs/py39/bin/python3.9
cachedir: .pytest_cache
rootdir: /Users/briford/work/dpkt, configfile: setup.cfg
plugins: cov-2.11.1, dash-1.19.0
collected 413 items

dpkt/__init__.py::test_utils PASSED                                      [  0%]
dpkt/__init__.py::test_getitem PASSED                                    [  0%]
dpkt/__init__.py::test_pack_hdr_overflow PASSED                          [  0%]
dpkt/__init__.py::test_pack_hdr_tuple PASSED                             [  0%]
dpkt/__init__.py::test_unpacking_failure PASSED                          [  1%]
dpkt/__init__.py::test_repr PASSED                                       [  1%]
dpkt/ah.py::test_default_creation PASSED                                 [  1%]
dpkt/ah.py::test_creation_from_buf PASSED                                [  1%]
dpkt/aim.py::testAIM PASSED                                              [  2%]
dpkt/aim.py::testExceptions PASSED                                       [  2%]
dpkt/aoe.py::test_creation PASSED                                        [  2%]
dpkt/aoe.py::test_properties PASSED                                      [  2%]
dpkt/aoe.py::test_unpack PASSED                                          [  3%]
dpkt/aoe.py::test_cmds PASSED                                            [  3%]
dpkt/aoe.py::test_cmd_loading PASSED                                     [  3%]
dpkt/aoeata.py::test_aoeata PASSED                                       [  3%]
dpkt/aoecfg.py::test_aoecfg PASSED                                       [  4%]
dpkt/asn1.py::test_asn1 PASSED                                           [  4%]
dpkt/asn1.py::test_utctime PASSED                                        [  4%]
dpkt/asn1.py::test_decode PASSED                                         [  4%]
dpkt/bgp.py::test_pack PASSED                                            [  5%]
dpkt/bgp.py::test_unpack PASSED                                          [  5%]
...
dpkt/tcp.py::test_offset PASSED                                          [ 97%]
dpkt/telnet.py::test_telnet PASSED                                       [ 98%]
dpkt/telnet.py::test_trailing_null PASSED                                [ 98%]
dpkt/tftp.py::test_op_rrq PASSED                                         [ 98%]
dpkt/tftp.py::test_op_data PASSED                                        [ 98%]
dpkt/tftp.py::test_op_err PASSED                                         [ 99%]
dpkt/tftp.py::test_op_other PASSED                                       [ 99%]
dpkt/tns.py::test_tns PASSED                                             [ 99%]
dpkt/utils.py::test_utils PASSED                                         [ 99%]
dpkt/vrrp.py::test_vrrp PASSED                                           [100%]

============================= 413 passed in 1.64s ==============================
kloczek commented 3 years ago

Please try latest pytest instead 9 months old one.

+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3
cachedir: .pytest_cache
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/.hypothesis/examples')
Using --randomly-seed=330444571
rootdir: /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2, configfile: setup.cfg
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, anyio-3.3.0, asyncio-0.15.1, trio-0.7.0, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0, Faker-8.12.1, nose2pytest-1.0.8, pyfakefs-4.5.1, tornado-0.8.1
collected 429 items / 2 errors / 427 selected

================================================================================== ERRORS ==================================================================================
________________________________________________________________________ ERROR collecting setup.py _________________________________________________________________________
/usr/lib64/python3.8/distutils/fancy_getopt.py:233: in getopt
    opts, args = getopt.getopt(args, short_opts, self.long_opts)
/usr/lib64/python3.8/getopt.py:95: in getopt
    opts, args = do_shorts(opts, args[0][1:], shortopts, args[1:])
/usr/lib64/python3.8/getopt.py:195: in do_shorts
    if short_has_arg(opt, shortopts):
/usr/lib64/python3.8/getopt.py:211: in short_has_arg
    raise GetoptError(_('option -%s not recognized') % opt, opt)
E   getopt.GetoptError: option -r not recognized

During handling of the above exception, another exception occurred:
/usr/lib64/python3.8/distutils/core.py:134: in setup
    ok = dist.parse_command_line()
/usr/lib64/python3.8/distutils/dist.py:475: in parse_command_line
    args = parser.getopt(args=self.script_args, object=self)
/usr/lib64/python3.8/distutils/fancy_getopt.py:235: in getopt
    raise DistutilsArgError(msg)
E   distutils.errors.DistutilsArgError: option -r not recognized

During handling of the above exception, another exception occurred:
setup.py:24: in <module>
    setup(name=package_name,
/usr/lib/python3.8/site-packages/setuptools/__init__.py:153: in setup
    return distutils.core.setup(**attrs)
/usr/lib64/python3.8/distutils/core.py:136: in setup
    raise SystemExit(gen_usage(dist.script_name) + "\nerror: %s" % msg)
E   SystemExit: usage: pytest [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
E      or: pytest --help [cmd1 cmd2 ...]
E      or: pytest --help-commands
E      or: pytest cmd --help
E
E   error: option -r not recognized
_______________________________________________________________ ERROR collecting examples/old/dhcprequest.py _______________________________________________________________
examples/old/dhcprequest.py:24: in <module>
    interface = pcapy.open_live(iface_name, 65536, 1, 0)
E   pcapy.PcapError: eno0: You don't have permission to capture on that device (socket: Operation not permitted)
----------------------------------------------------------------------------- Captured stdout ------------------------------------------------------------------------------
Auto Setting Interface to: eno0
========================================================================= short test summary info ==========================================================================
ERROR setup.py - SystemExit: usage: pytest [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
ERROR examples/old/dhcprequest.py - pcapy.PcapError: eno0: You don't have permission to capture on that device (socket: Operation not permitted)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================ 2 errors in 2.76s =============================================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.
brifordwylie commented 3 years ago

Errr.. I'm pretty sure this will work fine with any version of pytest.. but here's the latest (6.2.5)

> pytest --version
pytest 6.2.5

You can't run pytest without specifying the package directory (dpkt). If you just run pytest it will try to test ^everything^ including the setup.py which recursively tries to test setuptools/__init__.py which fails... so like I said before please run pytest dpkt.

> pytest dpkt
======================================== test session starts =========================================
platform darwin -- Python 3.9.6, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /Users/briford/.pyenv/versions/3.9.6/envs/py39/bin/python3.9
cachedir: .pytest_cache
rootdir: /Users/briford/work/dpkt, configfile: setup.cfg
plugins: cov-2.12.1, Faker-8.11.0, dash-1.21.0
collected 425 items

dpkt/__init__.py::test_utils PASSED                                                            [  0%]
dpkt/__init__.py::test_getitem_contains PASSED                                                 [  0%]
dpkt/__init__.py::test_pack_hdr_overflow PASSED                                                [  0%]
dpkt/__init__.py::test_bit_fields_overflow PASSED                                              [  0%]
dpkt/__init__.py::test_pack_hdr_tuple PASSED                                                   [  1%]
dpkt/__init__.py::test_unpacking_failure PASSED                                                [  1%]
dpkt/__init__.py::test_repr PASSED                                                             [  1%]
dpkt/ah.py::test_default_creation PASSED                                                       [  1%]
dpkt/ah.py::test_creation_from_buf PASSED
...

If you get error about coverage when running pytest dpkt you can install the code coverage packages: pip install pytest-cov coverage

kloczek commented 3 years ago

I have installed both modules

[tkloczko@barrel SPECS]$ pip show pytest-cov coverage
Name: pytest-cov
Version: 2.12.1
Summary: Pytest plugin for measuring coverage.
Home-page: https://github.com/pytest-dev/pytest-cov
Author: Marc Schlaich
Author-email: marc.schlaich@gmail.com
License: MIT
Location: /usr/lib/python3.8/site-packages
Requires: pytest, coverage, toml
Required-by:
---
Name: coverage
Version: 5.6b1
Summary: Code coverage measurement for Python
Home-page: https://github.com/nedbat/coveragepy
Author: Ned Batchelder and 142 others
Author-email: ned@nedbatchelder.com
License: Apache 2.0
Location: /usr/lib64/python3.8/site-packages
Requires:
Required-by: pytest-cov, nose2, coveralls, cov-core

Could you please repeat your test with -ra options? Looks like you are passing pytest options to other commands.

After add to ignore list those two files I have another failures:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-dpkt-1.9.7.2-2.1.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-dpkt-1.9.7.2-2.1.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -q --ignore examples/old/dhcprequest.py --ignore setup.py
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.11, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=3048967069
rootdir: /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2, configfile: setup.cfg
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, anyio-3.3.0, asyncio-0.15.1, trio-0.7.0, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0, Faker-8.12.1, nose2pytest-1.0.8, pyfakefs-4.5.1, tornado-0.8.1, twisted-1.13.3
collected 429 items

dpkt/aoeata.py .                                                                                                                                                     [  0%]
dpkt/ah.py ..                                                                                                                                                        [  0%]
dpkt/ethernet.py .....................                                                                                                                               [  5%]
dpkt/asn1.py ...                                                                                                                                                     [  6%]
dpkt/edp.py ........                                                                                                                                                 [  8%]
dpkt/llc.py ....                                                                                                                                                     [  9%]
dpkt/sll.py .                                                                                                                                                        [  9%]
dpkt/netbios.py .......                                                                                                                                              [ 10%]
dpkt/snoop.py .....................                                                                                                                                  [ 15%]
dpkt/sccp.py .                                                                                                                                                       [ 16%]
dpkt/pim.py .                                                                                                                                                        [ 16%]
examples/print_http_requests.py F                                                                                                                                    [ 16%]
dpkt/ssl_ciphersuites.py .......                                                                                                                                     [ 18%]
dpkt/radiotap.py .......                                                                                                                                             [ 19%]
examples/print_icmp.py F                                                                                                                                             [ 20%]
dpkt/stun.py ..                                                                                                                                                      [ 20%]
dpkt/pcapng.py ...................................                                                                                                                   [ 28%]
dpkt/gzip.py .............                                                                                                                                           [ 31%]
dpkt/bgp.py ...................                                                                                                                                      [ 36%]
dpkt/mrt.py .                                                                                                                                                        [ 36%]
dpkt/gre.py ......                                                                                                                                                   [ 37%]
dpkt/dhcp.py ..                                                                                                                                                      [ 38%]
dpkt/diameter.py ....                                                                                                                                                [ 39%]
dpkt/aoe.py .....                                                                                                                                                    [ 40%]
dpkt/stp.py ..                                                                                                                                                       [ 40%]
dpkt/ospf.py .                                                                                                                                                       [ 41%]
dpkt/ip.py ..........                                                                                                                                                [ 43%]
dpkt/ip6.py ................                                                                                                                                         [ 47%]
dpkt/tcp.py .....                                                                                                                                                    [ 48%]
dpkt/sll2.py .                                                                                                                                                       [ 48%]
dpkt/crc32c.py .                                                                                                                                                     [ 48%]
examples/print_dns_truncated.py F                                                                                                                                    [ 48%]
dpkt/utils.py .                                                                                                                                                      [ 49%]
dpkt/sctp.py ....                                                                                                                                                    [ 50%]
dpkt/tns.py .                                                                                                                                                        [ 50%]
dpkt/vrrp.py .                                                                                                                                                       [ 50%]
dpkt/__init__.py .......                                                                                                                                             [ 52%]
examples/print_packets.py F                                                                                                                                          [ 52%]
dpkt/ntp.py ..                                                                                                                                                       [ 52%]
dpkt/dns.py .........................................                                                                                                                [ 62%]
dpkt/rpc.py ......                                                                                                                                                   [ 63%]
dpkt/ppp.py .......                                                                                                                                                  [ 65%]
dpkt/rtp.py ..                                                                                                                                                       [ 65%]
dpkt/loopback.py ...                                                                                                                                                 [ 66%]
dpkt/igmp.py ..                                                                                                                                                      [ 67%]
dpkt/ieee80211.py ...............                                                                                                                                    [ 70%]
dpkt/ssl.py .........................................                                                                                                                [ 80%]
dpkt/aoecfg.py .                                                                                                                                                     [ 80%]
dpkt/smb.py .                                                                                                                                                        [ 80%]
dpkt/cdp.py ...                                                                                                                                                      [ 81%]
dpkt/http2.py ..............                                                                                                                                         [ 84%]
dpkt/aim.py ..                                                                                                                                                       [ 85%]
dpkt/rip.py .                                                                                                                                                        [ 85%]
dpkt/radius.py ...                                                                                                                                                   [ 86%]
dpkt/http.py ................                                                                                                                                        [ 89%]
dpkt/telnet.py ..                                                                                                                                                    [ 90%]
dpkt/netflow.py ..                                                                                                                                                   [ 90%]
dpkt/icmp.py ..                                                                                                                                                      [ 91%]
dpkt/tftp.py ....                                                                                                                                                    [ 92%]
dpkt/dpkt.py .......                                                                                                                                                 [ 93%]
dpkt/h225.py ....                                                                                                                                                    [ 94%]
dpkt/pcap.py ...............                                                                                                                                         [ 98%]
dpkt/pppoe.py .......                                                                                                                                                [ 99%]
dpkt/dtp.py .                                                                                                                                                        [100%]

================================================================================= FAILURES =================================================================================
___________________________________________________________________________________ test ___________________________________________________________________________________

    def test():
        """Open up a test pcap file and print out the packets"""
>       with open('data/http.pcap', 'rb') as f:
E       FileNotFoundError: [Errno 2] No such file or directory: 'data/http.pcap'

examples/print_http_requests.py:64: FileNotFoundError
___________________________________________________________________________________ test ___________________________________________________________________________________

    def test():
        """Open up a test pcap file and print out the packets"""
>       with open('data/dns_icmp.pcap', 'rb') as f:
E       FileNotFoundError: [Errno 2] No such file or directory: 'data/dns_icmp.pcap'

examples/print_icmp.py:51: FileNotFoundError
___________________________________________________________________________________ test ___________________________________________________________________________________

    def test():
        """Open up a test pcap file and print out the packets"""
>       with open('data/truncated_dns_2.pcap', 'rb') as f:
E       FileNotFoundError: [Errno 2] No such file or directory: 'data/truncated_dns_2.pcap'

examples/print_dns_truncated.py:92: FileNotFoundError
___________________________________________________________________________________ test ___________________________________________________________________________________

    def test():
        """Open up a test pcap file and print out the packets"""
>       with open('data/http.pcap', 'rb') as f:
E       FileNotFoundError: [Errno 2] No such file or directory: 'data/http.pcap'

examples/print_packets.py:46: FileNotFoundError
============================================================================= warnings summary =============================================================================
dpkt/ip.py::test_property_setters
  /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ip.py:128: UserWarning: IP.off is deprecated
    deprecation_warning("IP.off is deprecated")

dpkt/ip.py::test_property_setters
  /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ip.py:123: UserWarning: IP.off is deprecated
    deprecation_warning("IP.off is deprecated")

dpkt/ssl.py::TestServerHello::test_compression_method
  /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ssl.py:355: UserWarning: TLSServerHello.compression is deprecated and renamed to .compression_method
    deprecation_warning("TLSServerHello.compression is deprecated and renamed to .compression_method")

dpkt/ssl.py::TestServerHello::test_ciphersuite
  /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ssl.py:350: UserWarning: TLSServerHello.cipher_suite is deprecated and renamed to .ciphersuite
    deprecation_warning("TLSServerHello.cipher_suite is deprecated and renamed to .ciphersuite")

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================= short test summary info ==========================================================================
FAILED examples/print_http_requests.py::test - FileNotFoundError: [Errno 2] No such file or directory: 'data/http.pcap'
FAILED examples/print_icmp.py::test - FileNotFoundError: [Errno 2] No such file or directory: 'data/dns_icmp.pcap'
FAILED examples/print_dns_truncated.py::test - FileNotFoundError: [Errno 2] No such file or directory: 'data/truncated_dns_2.pcap'
FAILED examples/print_packets.py::test - FileNotFoundError: [Errno 2] No such file or directory: 'data/http.pcap'
================================================================ 4 failed, 425 passed, 4 warnings in 3.16s =================================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.

And I cannot find that data/http.pcap in source tree.

kloczek commented 3 years ago

And result with only testing dpkt directory:

+ /usr/bin/pytest -ra -q dpkt
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.11, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
Using --randomly-seed=3913316021
rootdir: /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2, configfile: setup.cfg
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, anyio-3.3.0, asyncio-0.15.1, trio-0.7.0, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, randomly-3.8.0, Faker-8.12.1, nose2pytest-1.0.8, pyfakefs-4.5.1, tornado-0.8.1, twisted-1.13.3
collected 425 items

dpkt/diameter.py ....                                                                                                                                                [  0%]
dpkt/dtp.py .                                                                                                                                                        [  1%]
dpkt/rip.py .                                                                                                                                                        [  1%]
dpkt/loopback.py ...                                                                                                                                                 [  2%]
dpkt/sctp.py ....                                                                                                                                                    [  3%]
dpkt/bgp.py ...................                                                                                                                                      [  7%]
dpkt/dns.py .........................................                                                                                                                [ 17%]
dpkt/sccp.py .                                                                                                                                                       [ 17%]
dpkt/radiotap.py .......                                                                                                                                             [ 19%]
dpkt/rtp.py ..                                                                                                                                                       [ 19%]
dpkt/utils.py .                                                                                                                                                      [ 19%]
dpkt/ethernet.py .....................                                                                                                                               [ 24%]
dpkt/crc32c.py .                                                                                                                                                     [ 24%]
dpkt/pppoe.py .......                                                                                                                                                [ 26%]
dpkt/snoop.py ..........F..........                                                                                                                                  [ 31%]
dpkt/http2.py ..............                                                                                                                                         [ 34%]
dpkt/stp.py ..                                                                                                                                                       [ 35%]
dpkt/sll2.py .                                                                                                                                                       [ 35%]
dpkt/stun.py ..                                                                                                                                                      [ 36%]
dpkt/http.py ................                                                                                                                                        [ 39%]
dpkt/cdp.py ...                                                                                                                                                      [ 40%]
dpkt/llc.py ....                                                                                                                                                     [ 41%]
dpkt/ssl.py .........................................                                                                                                                [ 51%]
dpkt/radius.py ...                                                                                                                                                   [ 51%]
dpkt/netflow.py ..                                                                                                                                                   [ 52%]
dpkt/aoecfg.py .                                                                                                                                                     [ 52%]
dpkt/telnet.py ..                                                                                                                                                    [ 52%]
dpkt/sll.py .                                                                                                                                                        [ 53%]
dpkt/rpc.py ......                                                                                                                                                   [ 54%]
dpkt/h225.py ....                                                                                                                                                    [ 55%]
dpkt/pcapng.py ...................................                                                                                                                   [ 63%]
dpkt/aim.py ..                                                                                                                                                       [ 64%]
dpkt/pim.py .                                                                                                                                                        [ 64%]
dpkt/ntp.py ..                                                                                                                                                       [ 64%]
dpkt/pcap.py ...............                                                                                                                                         [ 68%]
dpkt/ieee80211.py ...............                                                                                                                                    [ 72%]
dpkt/asn1.py ...                                                                                                                                                     [ 72%]
dpkt/smb.py .                                                                                                                                                        [ 72%]
dpkt/ip6.py ................                                                                                                                                         [ 76%]
dpkt/mrt.py .                                                                                                                                                        [ 76%]
dpkt/aoe.py .....                                                                                                                                                    [ 78%]
dpkt/gre.py ......                                                                                                                                                   [ 79%]
dpkt/tftp.py ....                                                                                                                                                    [ 80%]
dpkt/gzip.py .............                                                                                                                                           [ 83%]
dpkt/icmp.py ..                                                                                                                                                      [ 84%]
dpkt/edp.py ........                                                                                                                                                 [ 85%]
dpkt/dpkt.py .......                                                                                                                                                 [ 87%]
dpkt/ip.py ..........                                                                                                                                                [ 89%]
dpkt/ah.py ..                                                                                                                                                        [ 90%]
dpkt/tcp.py .....                                                                                                                                                    [ 91%]
dpkt/vrrp.py .                                                                                                                                                       [ 91%]
dpkt/ssl_ciphersuites.py .......                                                                                                                                     [ 93%]
dpkt/netbios.py .......                                                                                                                                              [ 95%]
dpkt/aoeata.py .                                                                                                                                                     [ 95%]
dpkt/tns.py .                                                                                                                                                        [ 95%]
dpkt/ospf.py .                                                                                                                                                       [ 95%]
dpkt/ppp.py .......                                                                                                                                                  [ 97%]
dpkt/igmp.py ..                                                                                                                                                      [ 97%]
dpkt/__init__.py .......                                                                                                                                             [ 99%]
dpkt/dhcp.py ..                                                                                                                                                      [100%]

================================================================================= FAILURES =================================================================================
______________________________________________________________ TestSnoopWriter.test_snoop_file_writer_filehdr ______________________________________________________________

self = <dpkt.snoop.TestSnoopWriter object at 0x7f3a2b553310>

    def test_snoop_file_writer_filehdr(self):
        # jump to the start and read the file header
>       self.fobj.seek(0)
E       ValueError: I/O operation on closed file.

dpkt/snoop.py:351: ValueError
============================================================================= warnings summary =============================================================================
dpkt/ssl.py::TestServerHello::test_ciphersuite
  /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ssl.py:350: UserWarning: TLSServerHello.cipher_suite is deprecated and renamed to .ciphersuite
    deprecation_warning("TLSServerHello.cipher_suite is deprecated and renamed to .ciphersuite")

dpkt/ssl.py::TestServerHello::test_compression_method
  /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ssl.py:355: UserWarning: TLSServerHello.compression is deprecated and renamed to .compression_method
    deprecation_warning("TLSServerHello.compression is deprecated and renamed to .compression_method")

dpkt/ip.py::test_property_setters
  /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ip.py:128: UserWarning: IP.off is deprecated
    deprecation_warning("IP.off is deprecated")

dpkt/ip.py::test_property_setters
  /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ip.py:123: UserWarning: IP.off is deprecated
    deprecation_warning("IP.off is deprecated")

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================= short test summary info ==========================================================================
FAILED dpkt/snoop.py::TestSnoopWriter::test_snoop_file_writer_filehdr - ValueError: I/O operation on closed file.
================================================================ 1 failed, 424 passed, 4 warnings in 3.12s =================================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.
kloczek commented 3 years ago

Second try and I have another unit popping out. Looks like there are some test units dependencies and because I'm using pytest -randomly it exposes that kind of issues.

kloczek commented 3 years ago

Yep running pytest with disabled randomly plugin is fine. Still there are some warnings:

+ /usr/bin/pytest -ra -q dpkt -p no:randomly
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.11, pytest-6.2.5, py-1.10.0, pluggy-0.13.1
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2, configfile: setup.cfg
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, expect-1.1.0, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, freezegun-0.4.2, aspectlib-1.5.2, toolbox-0.5, rerunfailures-9.1.1, requests-mock-1.9.3, cov-2.12.1, flaky-3.7.0, benchmark-3.4.1, xdist-2.3.0, pylama-7.7.1, datadir-1.3.1, regressions-2.2.0, cases-3.6.3, xprocess-0.18.1, black-0.3.12, anyio-3.3.0, asyncio-0.15.1, trio-0.7.0, subtests-0.5.0, isort-2.0.0, hypothesis-6.14.6, mock-3.6.1, profiling-1.7.0, Faker-8.12.1, nose2pytest-1.0.8, pyfakefs-4.5.1, tornado-0.8.1, twisted-1.13.3
collected 425 items

dpkt/__init__.py .......                                                                                                                                             [  1%]
dpkt/ah.py ..                                                                                                                                                        [  2%]
dpkt/aim.py ..                                                                                                                                                       [  2%]
dpkt/aoe.py .....                                                                                                                                                    [  3%]
dpkt/aoeata.py .                                                                                                                                                     [  4%]
dpkt/aoecfg.py .                                                                                                                                                     [  4%]
dpkt/asn1.py ...                                                                                                                                                     [  4%]
dpkt/bgp.py ...................                                                                                                                                      [  9%]
dpkt/cdp.py ...                                                                                                                                                      [ 10%]
dpkt/crc32c.py .                                                                                                                                                     [ 10%]
dpkt/dhcp.py ..                                                                                                                                                      [ 10%]
dpkt/diameter.py ....                                                                                                                                                [ 11%]
dpkt/dns.py .........................................                                                                                                                [ 21%]
dpkt/dpkt.py .......                                                                                                                                                 [ 23%]
dpkt/dtp.py .                                                                                                                                                        [ 23%]
dpkt/edp.py ........                                                                                                                                                 [ 25%]
dpkt/ethernet.py .....................                                                                                                                               [ 30%]
dpkt/gre.py ......                                                                                                                                                   [ 31%]
dpkt/gzip.py .............                                                                                                                                           [ 34%]
dpkt/h225.py ....                                                                                                                                                    [ 35%]
dpkt/http.py ................                                                                                                                                        [ 39%]
dpkt/http2.py ..............                                                                                                                                         [ 42%]
dpkt/icmp.py ..                                                                                                                                                      [ 43%]
dpkt/ieee80211.py ...............                                                                                                                                    [ 46%]
dpkt/igmp.py ..                                                                                                                                                      [ 47%]
dpkt/ip.py ..........                                                                                                                                                [ 49%]
dpkt/ip6.py ................                                                                                                                                         [ 53%]
dpkt/llc.py ....                                                                                                                                                     [ 54%]
dpkt/loopback.py ...                                                                                                                                                 [ 54%]
dpkt/mrt.py .                                                                                                                                                        [ 55%]
dpkt/netbios.py .......                                                                                                                                              [ 56%]
dpkt/netflow.py ..                                                                                                                                                   [ 57%]
dpkt/ntp.py ..                                                                                                                                                       [ 57%]
dpkt/ospf.py .                                                                                                                                                       [ 57%]
dpkt/pcap.py ...............                                                                                                                                         [ 61%]
dpkt/pcapng.py ...................................                                                                                                                   [ 69%]
dpkt/pim.py .                                                                                                                                                        [ 69%]
dpkt/ppp.py .......                                                                                                                                                  [ 71%]
dpkt/pppoe.py .......                                                                                                                                                [ 73%]
dpkt/radiotap.py .......                                                                                                                                             [ 74%]
dpkt/radius.py ...                                                                                                                                                   [ 75%]
dpkt/rip.py .                                                                                                                                                        [ 75%]
dpkt/rpc.py ......                                                                                                                                                   [ 77%]
dpkt/rtp.py ..                                                                                                                                                       [ 77%]
dpkt/sccp.py .                                                                                                                                                       [ 77%]
dpkt/sctp.py ....                                                                                                                                                    [ 78%]
dpkt/sll.py .                                                                                                                                                        [ 79%]
dpkt/sll2.py .                                                                                                                                                       [ 79%]
dpkt/smb.py .                                                                                                                                                        [ 79%]
dpkt/snoop.py .....................                                                                                                                                  [ 84%]
dpkt/ssl.py .........................................                                                                                                                [ 94%]
dpkt/ssl_ciphersuites.py .......                                                                                                                                     [ 95%]
dpkt/stp.py ..                                                                                                                                                       [ 96%]
dpkt/stun.py ..                                                                                                                                                      [ 96%]
dpkt/tcp.py .....                                                                                                                                                    [ 97%]
dpkt/telnet.py ..                                                                                                                                                    [ 98%]
dpkt/tftp.py ....                                                                                                                                                    [ 99%]
dpkt/tns.py .                                                                                                                                                        [ 99%]
dpkt/utils.py .                                                                                                                                                      [ 99%]
dpkt/vrrp.py .                                                                                                                                                       [100%]

============================================================================= warnings summary =============================================================================
dpkt/ip.py::test_property_setters
  /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ip.py:128: UserWarning: IP.off is deprecated
    deprecation_warning("IP.off is deprecated")

dpkt/ip.py::test_property_setters
  /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ip.py:123: UserWarning: IP.off is deprecated
    deprecation_warning("IP.off is deprecated")

dpkt/ssl.py::TestServerHello::test_ciphersuite
  /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ssl.py:350: UserWarning: TLSServerHello.cipher_suite is deprecated and renamed to .ciphersuite
    deprecation_warning("TLSServerHello.cipher_suite is deprecated and renamed to .ciphersuite")

dpkt/ssl.py::TestServerHello::test_compression_method
  /home/tkloczko/rpmbuild/BUILD/dpkt-1.9.7.2/dpkt/ssl.py:355: UserWarning: TLSServerHello.compression is deprecated and renamed to .compression_method
    deprecation_warning("TLSServerHello.compression is deprecated and renamed to .compression_method")

-- Docs: https://docs.pytest.org/en/stable/warnings.html
===================================================================== 425 passed, 4 warnings in 2.52s ======================================================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.

units dependencies issues can be solved using https://github.com/mrbean-bremen/pytest-find-dependencies/

brifordwylie commented 3 years ago

@kloczek if you want to run the examples you can simply run those by

> cd examples
> python print_http_requests.py
Timestamp:  2004-05-13 10:17:08.222534
Ethernet Frame:  00:00:01:00:00:00 fe:ff:20:00:01:00 2048
IP: 145.254.160.237 -> 65.208.228.223   (len=519 ttl=128 DF=1 MF=0 offset=0)
HTTP request: Request(version='1.1', method='GET', uri='/download.html', headers=OrderedDict([('host', 'www.ethereal.com'), ('user-agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113'), ('accept', '
...

The data directory for examples is in examples/data

As stated before you can simply run pytest dpkt. Github automatically runs pytest on every PR for different versions of python and it's working fine. See .github/workflows/python-package.yml. Supporting pytest-randomly isn't going to be something that the maintainers (which are already stretched thin) are going to get to anytime soon. If you really want pytest-randomly supported please submit a PR.

kloczek commented 3 years ago

@kloczek if you want to run the examples you can simply run those by

Nope I don't want to have that embedded in packaging procedure. Nevertheless if some files or directories should be excluded from collecting test units it should be marked in pytest settings :)