pynetwork / pypcap

pypcap - python libpcap module, forked from code.google.com/p/pypcap
Other
299 stars 75 forks source link

MacOS Big Sur unable to install #104

Open phunkyg opened 3 years ago

phunkyg commented 3 years ago

Attempting to use pypcap under python 2.7 installing via pip.

None of the following found: ['libpcap.a', 'libpcap.so', 'libpcap.dylib', 'wpcap.lib']

Appears to be related to the changes introduced in Big Sur that means the .dylib files no longer exist, they are all part of the new dynamic cache.

For example, here is the homebrew project having the same: https://github.com/Homebrew/brew/issues/7803 Another: https://github.com/openframeworks/openFrameworks/issues/6599#issuecomment-691685881 More: https://mjtsai.com/blog/2020/06/26/reverse-engineering-macos-11-0/

The Apple release note having the info: https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-release-notes

Understandably being as Apple are moving things and changing the norms, lots of this kind of thing are going to break. Looking for a workaround if possible or a fix that implements the dlopen() alternative - whatever that means!

New in macOS Big Sur 11.0.1, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache. (62986286)
guyharris commented 3 years ago

In

New in macOS Big Sur 11.0.1, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache. (62986286)

the dlopen() alternative is "you attempt to determine whether you can use the library by attempting to use the library, rather than by some other technique making an assumption about how the library is implemented".

So one alternative would be to modify find_lib_path_and_file() to use dlopen() on most UN*Xes (with the exception of 32-bit HP-UX) and LoadLibrary() on Windows. Older versions of some UN*Xes might not support dlopen()ing shared libraries (Mach-O has separate file types for shared libraries and dynamically-loaded files - newer versions of macOS treat them the same, but older versions may treat them differently), so both attempting to load the library through the official system API and looking for the file might be necessary.

(Can ctypes be used?)

Another alternative would be "Apple's shipped libpcap and tcpdump since Day One, or close to it, so we'll just assume we have it". If that's too risky, you could try running the command

otool -L `which tcpdump`

from the setup script if you're on macOS, and capture its output, to 1) find tcpdump and 2) get the paths of the libraries it has to load; on my 11.2.3 (x86-64) VM, it prints:

/usr/sbin/tcpdump:
        /usr/lib/libssl.46.dylib (compatibility version 47.0.0, current version 47.1.0)
        /usr/lib/libcrypto.44.dylib (compatibility version 45.0.0, current version 45.1.0)
        /usr/lib/libpcap.A.dylib (compatibility version 1.0.0, current version 1.0.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
phunkyg commented 3 years ago

Another alternative would be "Apple's shipped libpcap and tcpdump since Day One, or close to it, so we'll just assume we have it". If that's too risky, you could try running the command

Confirmed. Ignoring the absence of these files during setup.py allows installation to continue, then everything seems (superficially) OK.

In setup.py, commented out lines 87, 88

Added lib_file_path = '/usr/lib'

mwfong-csl commented 2 years ago

Here's a version of setup.py that uses ctypes to address this issue. If it works, I can create a merge request.

setup_py.zip

mwfong-csl commented 2 years ago

I've created a fork, issue-104, that incorporates @hellais's suggestions. However, because my Mac isn't running Big Sur, could someone please test? Once that's done, I'll issue a PR.

slhck commented 2 years ago

I still cannot install, at least under macOS 12.5.1. The error is:

➜ git clone https://github.com/pynetwork/pypcap/
➜ cd pypcap
➜ pip3 install .
Processing /Users/werner/Documents/Software/pypcap
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [2 lines of output]
      Found pcap headers in /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/pcap.h
      None of the following found: ['libpcap.a', 'libpcap.so', 'libpcap.dylib', 'wpcap.lib']
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details

This is despite also setting the following as suggested by Homebrew:

export LDFLAGS="-L/opt/homebrew/opt/libpcap/lib"
export CPPFLAGS="-I/opt/homebrew/opt/libpcap/include"
export PKG_CONFIG_PATH="/opt/homebrew/opt/libpcap/lib/pkgconfig"

I cannot test for Big Sur, but the issue is the same.

koczanm commented 2 years ago

@mwfong-csl I've just tested your change and can confirm it works as expected under macOS 12.5.1 Monterey. Thanks for making it.

As a confirmation, here's the build output:

pypcap 🍣 issue-104 🛤️ ×1 via C v13.1.6-clang 🐍 v3.8.13
🕙 20:52:48 ❯ py setup.py install
Found pcap headers in /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/pcap.h
Found libraries in /usr/lib/libpcap.dylib
found pcap_get_tstamp_precision function
found pcap_setdirection
found pcap_setnonblock
found pcap_compile_nopcap function
/Users/koczanm/.asdf/installs/python/3.8.13/lib/python3.8/site-packages/setuptools/dist.py:642: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
  warnings.warn(
running install
running bdist_egg
running egg_info
creating pypcap.egg-info
writing pypcap.egg-info/PKG-INFO
writing dependency_links to pypcap.egg-info/dependency_links.txt
writing top-level names to pypcap.egg-info/top_level.txt
writing manifest file 'pypcap.egg-info/SOURCES.txt'
adding license file 'LICENSE' (matched pattern 'LICEN[CS]E*')
adding license file 'AUTHORS' (matched pattern 'AUTHORS*')
reading manifest file 'pypcap.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pypcap.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-12.5-arm64/egg
running install_lib
running build_ext
building 'pcap' extension
creating build
creating build/temp.macosx-12.5-arm64-3.8
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -DHAVE_PCAP_TSTAMP_PRECISION=1 -DHAVE_PCAP_SETDIRECTION=1 -DHAVE_PCAP_SETNONBLOCK=1 -DHAVE_PCAP_COMPILE_NOPCAP=1 -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include -I/Users/koczanm/.asdf/installs/python/3.8.13/include/python3.8 -c pcap.c -o build/temp.macosx-12.5-arm64-3.8/pcap.o
208 warnings generated.
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -DHAVE_PCAP_TSTAMP_PRECISION=1 -DHAVE_PCAP_SETDIRECTION=1 -DHAVE_PCAP_SETNONBLOCK=1 -DHAVE_PCAP_COMPILE_NOPCAP=1 -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include -I/Users/koczanm/.asdf/installs/python/3.8.13/include/python3.8 -c pcap_ex.c -o build/temp.macosx-12.5-arm64-3.8/pcap_ex.o
37 warnings generated.
creating build/lib.macosx-12.5-arm64-3.8
clang -bundle -undefined dynamic_lookup -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/koczanm/.asdf/installs/python/3.8.13/lib -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/koczanm/.asdf/installs/python/3.8.13/lib -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib build/temp.macosx-12.5-arm64-3.8/pcap.o build/temp.macosx-12.5-arm64-3.8/pcap_ex.o -L/usr/lib -lpcap -o build/lib.macosx-12.5-arm64-3.8/pcap.cpython-38-darwin.so
creating build/bdist.macosx-12.5-arm64
creating build/bdist.macosx-12.5-arm64/egg
copying build/lib.macosx-12.5-arm64-3.8/pcap.cpython-38-darwin.so -> build/bdist.macosx-12.5-arm64/egg
creating stub loader for pcap.cpython-38-darwin.so
byte-compiling build/bdist.macosx-12.5-arm64/egg/pcap.py to pcap.cpython-38.pyc
creating build/bdist.macosx-12.5-arm64/egg/EGG-INFO
copying pypcap.egg-info/PKG-INFO -> build/bdist.macosx-12.5-arm64/egg/EGG-INFO
copying pypcap.egg-info/SOURCES.txt -> build/bdist.macosx-12.5-arm64/egg/EGG-INFO
copying pypcap.egg-info/dependency_links.txt -> build/bdist.macosx-12.5-arm64/egg/EGG-INFO
copying pypcap.egg-info/top_level.txt -> build/bdist.macosx-12.5-arm64/egg/EGG-INFO
writing build/bdist.macosx-12.5-arm64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
__pycache__.pcap.cpython-38: module references __file__
creating dist
creating 'dist/pypcap-1.3.0-py3.8-macosx-12.5-arm64.egg' and adding 'build/bdist.macosx-12.5-arm64/egg' to it
removing 'build/bdist.macosx-12.5-arm64/egg' (and everything under it)
Processing pypcap-1.3.0-py3.8-macosx-12.5-arm64.egg
creating /Users/koczanm/.asdf/installs/python/3.8.13/lib/python3.8/site-packages/pypcap-1.3.0-py3.8-macosx-12.5-arm64.egg
Extracting pypcap-1.3.0-py3.8-macosx-12.5-arm64.egg to /Users/koczanm/.asdf/installs/python/3.8.13/lib/python3.8/site-packages
Adding pypcap 1.3.0 to easy-install.pth file

Installed /Users/koczanm/.asdf/installs/python/3.8.13/lib/python3.8/site-packages/pypcap-1.3.0-py3.8-macosx-12.5-arm64.egg
Processing dependencies for pypcap==1.3.0
Finished processing dependencies for pypcap==1.3.0

There were some warnings from clang but I skip them to keep the output as short as I can.

slhck commented 2 years ago

How did you install libpcap? Seems it's in a system wide location.

I guess the reason it doesn't work for me is that it's the Homebrew library, but even setting the additional paths won't work.

koczanm commented 2 years ago

@mwfong-csl's change takes advantage of ctypes.util.find_library() (for more info see this PR). So even though /usr/lib/libpcap.dylib doesn't actually exists, it's returned by the function, as below:

pypcap 🍣 issue-104 🛤️ ×1 via C v13.1.6-clang 🐍 v3.8.13
🕙 21:03:42 ❯ ls /usr/lib/libpcap.dylib
lsd: /usr/lib/libpcap.dylib: No such file or directory (os error 2).

pypcap 🍣 issue-104 🛤️ ×1 via C v13.1.6-clang 🐍 v3.8.13
🕙 21:03:54 ❯ py -c "from ctypes.util import find_library; print(find_library('libpcap.dylib'))"
/usr/lib/libpcap.dylib
mwfong-csl commented 2 years ago

@slhck Per ldconfig equivalent for Mac OS X, did you specify the Homebrew library path via the environment symbol, DYLD_LIBRARY_PATH?

slhck commented 2 years ago
➜ ll /opt/homebrew/opt/libpcap/lib
total 1128
-rw-r--r--  1 werner  admin  268208 Nov  8  2021 libpcap.1.10.1.dylib
lrwxr-xr-x  1 werner  admin      20 Jun  7  2021 libpcap.A.dylib -> libpcap.1.10.1.dylib
-r--r--r--  1 werner  admin  305192 Jun  7  2021 libpcap.a
lrwxr-xr-x  1 werner  admin      15 Jun  7  2021 libpcap.dylib -> libpcap.A.dylib
drwxr-xr-x  3 werner  admin      96 Nov  8  2021 pkgconfig
➜ DYLD_LIBRARY_PATH=/opt/homebrew/opt/libpcap/lib:$DYLD_LIBRARY_PATH pip3 install .
Processing /Users/werner/Documents/Software/pypcap
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [2 lines of output]
      Found pcap headers in /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/pcap.h
      None of the following found: ['libpcap.a', 'libpcap.so', 'libpcap.dylib', 'wpcap.lib']
      [end of output]

I'm not that experienced with building/shipping these projects in a platform-independent manner, but wouldn't it be possible to rely on pkg-config to locate the headers and libraries rather than attempting to manually list all possible locations? libpcap installs pkg-config files under Linux and so does the version shipped with Homebrew.

tulay commented 2 years ago

I tested @mwfong-csl's branch on a clean install Big Sur VM with standard libraries (not brew install) and tested with py36, py37, py38, py39, py310. The installation succeeded on python versions >= py3.8. Here is the output:

tmac@Tests-Mac pypcap % tox -e py36,py37,py38,py39,py310
GLOB sdist-make: /Users/tmac/github-projects/mfong/pypcap/setup.py
py36 inst-nodeps: /Users/tmac/github-projects/mfong/pypcap/.tox/.tmp/package/1/pypcap-1.3.0.zip
ERROR: invocation failed (exit code 1), logfile: /Users/tmac/github-projects/mfong/pypcap/.tox/py36/log/py36-3.log
================================================================= log start ==================================================================
Processing ./.tox/.tmp/package/1/pypcap-1.3.0.zip
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /Users/tmac/github-projects/mfong/pypcap/.tox/py36/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/9z/rfwzqhzn31v0_q6xz7bq59m00000gn/T/pip-req-build-hf6ao2pc/setup.py'"'"'; __file__='"'"'/private/var/folders/9z/rfwzqhzn31v0_q6xz7bq59m00000gn/T/pip-req-build-hf6ao2pc/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/9z/rfwzqhzn31v0_q6xz7bq59m00000gn/T/pip-pip-egg-info-jvmzev3o
       cwd: /private/var/folders/9z/rfwzqhzn31v0_q6xz7bq59m00000gn/T/pip-req-build-hf6ao2pc/
  Complete output (2 lines):
  Found pcap headers in /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk/usr/include/pcap.h
  None of the following found: ['libpcap.a', 'libpcap.so', 'libpcap.dylib', 'wpcap.lib']
  ----------------------------------------
WARNING: Discarding file:///Users/tmac/github-projects/mfong/pypcap/.tox/.tmp/package/1/pypcap-1.3.0.zip. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

================================================================== log end ===================================================================
py37 inst-nodeps: /Users/tmac/github-projects/mfong/pypcap/.tox/.tmp/package/1/pypcap-1.3.0.zip
ERROR: invocation failed (exit code 1), logfile: /Users/tmac/github-projects/mfong/pypcap/.tox/py37/log/py37-3.log
================================================================= log start ==================================================================
Processing ./.tox/.tmp/package/1/pypcap-1.3.0.zip
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'error'
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [2 lines of output]
      Found pcap headers in /Library/Developer/CommandLineTools/SDKs/MacOSX11.1.sdk/usr/include/pcap.h
      None of the following found: ['libpcap.a', 'libpcap.so', 'libpcap.dylib', 'wpcap.lib']
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

================================================================== log end ===================================================================
py38 inst-nodeps: /Users/tmac/github-projects/mfong/pypcap/.tox/.tmp/package/1/pypcap-1.3.0.zip
py38 installed: attrs==22.1.0,coverage==6.4.4,Cython==0.29.32,dpkt==1.9.8,iniconfig==1.1.1,packaging==21.3,pluggy==1.0.0,py==1.11.0,pyparsing==3.0.9,pypcap @ file:///Users/tmac/github-projects/mfong/pypcap/.tox/.tmp/package/1/pypcap-1.3.0.zip,pytest==7.1.3,pytest-cov==3.0.0,tomli==2.0.1
py38 run-test-pre: PYTHONHASHSEED='1348822243'
py38 run-test: commands[0] | pip install -e .
Obtaining file:///Users/tmac/github-projects/mfong/pypcap
  Preparing metadata (setup.py) ... done
Installing collected packages: pypcap
  Attempting uninstall: pypcap
    Found existing installation: pypcap 1.3.0
    Uninstalling pypcap-1.3.0:
      Successfully uninstalled pypcap-1.3.0
  Running setup.py develop for pypcap
Successfully installed pypcap-1.3.0
py38 run-test: commands[1] | pytest tests --cov=tests
============================================================ test session starts =============================================================
platform darwin -- Python 3.8.13, pytest-7.1.3, pluggy-1.0.0 -- /Users/tmac/github-projects/mfong/pypcap/.tox/py38/bin/python
cachedir: .tox/py38/.pytest_cache
rootdir: /Users/tmac/github-projects/mfong/pypcap, configfile: setup.cfg
plugins: cov-3.0.0
collected 11 items                                                                                                                           

tests/test.py::test_pcap_iter PASSED                                                                                                   [  9%]
tests/test.py::test_pcap_iter_ns PASSED                                                                                                [ 18%]
tests/test.py::test_pcap_properties PASSED                                                                                             [ 27%]
tests/test.py::test_pcap_errors PASSED                                                                                                 [ 36%]
tests/test.py::test_pcap_dispatch PASSED                                                                                               [ 45%]
tests/test.py::test_pcap_dispatch_ns PASSED                                                                                            [ 54%]
tests/test.py::test_pcap_dispatch_exception PASSED                                                                                     [ 63%]
tests/test.py::test_pcap_readpkts PASSED                                                                                               [ 72%]
tests/test.py::test_pcap_overwritten PASSED                                                                                            [ 81%]
tests/test.py::test_pcap_loop_overwritten PASSED                                                                                       [ 90%]
tests/test.py::test_unicode PASSED                                                                                                     [100%]

---------- coverage: platform darwin, python 3.8.13-final-0 ----------
Name                 Stmts   Miss  Cover   Missing
--------------------------------------------------
tests/test.py          137     17    88%   156-157, 177, 179, 196-198, 202-212
tests/testsniff.py      48     38    21%   11-21, 25-29, 33-34, 38-40, 44-75, 78
--------------------------------------------------
TOTAL                  185     55    70%

============================================================= 11 passed in 1.74s =============================================================
py39 inst-nodeps: /Users/tmac/github-projects/mfong/pypcap/.tox/.tmp/package/1/pypcap-1.3.0.zip
py39 installed: attrs==22.1.0,coverage==6.4.4,Cython==0.29.32,dpkt==1.9.8,iniconfig==1.1.1,packaging==21.3,pluggy==1.0.0,py==1.11.0,pyparsing==3.0.9,pypcap @ file:///Users/tmac/github-projects/mfong/pypcap/.tox/.tmp/package/1/pypcap-1.3.0.zip,pytest==7.1.3,pytest-cov==3.0.0,tomli==2.0.1
py39 run-test-pre: PYTHONHASHSEED='1348822243'
py39 run-test: commands[0] | pip install -e .
Obtaining file:///Users/tmac/github-projects/mfong/pypcap
  Preparing metadata (setup.py) ... done
Installing collected packages: pypcap
  Attempting uninstall: pypcap
    Found existing installation: pypcap 1.3.0
    Uninstalling pypcap-1.3.0:
      Successfully uninstalled pypcap-1.3.0
  Running setup.py develop for pypcap
Successfully installed pypcap-1.3.0
py39 run-test: commands[1] | pytest tests --cov=tests
============================================================ test session starts =============================================================
platform darwin -- Python 3.9.13, pytest-7.1.3, pluggy-1.0.0 -- /Users/tmac/github-projects/mfong/pypcap/.tox/py39/bin/python
cachedir: .tox/py39/.pytest_cache
rootdir: /Users/tmac/github-projects/mfong/pypcap, configfile: setup.cfg
plugins: cov-3.0.0
collected 11 items                                                                                                                           

tests/test.py::test_pcap_iter PASSED                                                                                                   [  9%]
tests/test.py::test_pcap_iter_ns PASSED                                                                                                [ 18%]
tests/test.py::test_pcap_properties PASSED                                                                                             [ 27%]
tests/test.py::test_pcap_errors PASSED                                                                                                 [ 36%]
tests/test.py::test_pcap_dispatch PASSED                                                                                               [ 45%]
tests/test.py::test_pcap_dispatch_ns PASSED                                                                                            [ 54%]
tests/test.py::test_pcap_dispatch_exception PASSED                                                                                     [ 63%]
tests/test.py::test_pcap_readpkts PASSED                                                                                               [ 72%]
tests/test.py::test_pcap_overwritten PASSED                                                                                            [ 81%]
tests/test.py::test_pcap_loop_overwritten PASSED                                                                                       [ 90%]
tests/test.py::test_unicode PASSED                                                                                                     [100%]

---------- coverage: platform darwin, python 3.9.13-final-0 ----------
Name                 Stmts   Miss  Cover   Missing
--------------------------------------------------
tests/test.py          137     17    88%   156-157, 177, 179, 196-198, 202-212
tests/testsniff.py      48     38    21%   11-21, 25-29, 33-34, 38-40, 44-75, 78
--------------------------------------------------
TOTAL                  185     55    70%

============================================================= 11 passed in 1.75s =============================================================
py310 inst-nodeps: /Users/tmac/github-projects/mfong/pypcap/.tox/.tmp/package/1/pypcap-1.3.0.zip
py310 installed: attrs==22.1.0,coverage==6.4.4,Cython==0.29.32,dpkt==1.9.8,iniconfig==1.1.1,packaging==21.3,pluggy==1.0.0,py==1.11.0,pyparsing==3.0.9,pypcap @ file:///Users/tmac/github-projects/mfong/pypcap/.tox/.tmp/package/1/pypcap-1.3.0.zip,pytest==7.1.3,pytest-cov==3.0.0,tomli==2.0.1
py310 run-test-pre: PYTHONHASHSEED='1348822243'
py310 run-test: commands[0] | pip install -e .
Obtaining file:///Users/tmac/github-projects/mfong/pypcap
  Preparing metadata (setup.py) ... done
Installing collected packages: pypcap
  Attempting uninstall: pypcap
    Found existing installation: pypcap 1.3.0
    Uninstalling pypcap-1.3.0:
      Successfully uninstalled pypcap-1.3.0
  Running setup.py develop for pypcap
Successfully installed pypcap-1.3.0
py310 run-test: commands[1] | pytest tests --cov=tests
============================================================ test session starts =============================================================
platform darwin -- Python 3.10.6, pytest-7.1.3, pluggy-1.0.0 -- /Users/tmac/github-projects/mfong/pypcap/.tox/py310/bin/python
cachedir: .tox/py310/.pytest_cache
rootdir: /Users/tmac/github-projects/mfong/pypcap, configfile: setup.cfg
plugins: cov-3.0.0
collected 11 items                                                                                                                           

tests/test.py::test_pcap_iter PASSED                                                                                                   [  9%]
tests/test.py::test_pcap_iter_ns PASSED                                                                                                [ 18%]
tests/test.py::test_pcap_properties PASSED                                                                                             [ 27%]
tests/test.py::test_pcap_errors PASSED                                                                                                 [ 36%]
tests/test.py::test_pcap_dispatch PASSED                                                                                               [ 45%]
tests/test.py::test_pcap_dispatch_ns PASSED                                                                                            [ 54%]
tests/test.py::test_pcap_dispatch_exception PASSED                                                                                     [ 63%]
tests/test.py::test_pcap_readpkts PASSED                                                                                               [ 72%]
tests/test.py::test_pcap_overwritten PASSED                                                                                            [ 81%]
tests/test.py::test_pcap_loop_overwritten PASSED                                                                                       [ 90%]
tests/test.py::test_unicode PASSED                                                                                                     [100%]

---------- coverage: platform darwin, python 3.10.6-final-0 ----------
Name                 Stmts   Miss  Cover   Missing
--------------------------------------------------
tests/test.py          137     17    88%   156-157, 177, 179, 196-198, 202-212
tests/testsniff.py      48     38    21%   11-21, 25-29, 33-34, 38-40, 44-75, 78
--------------------------------------------------
TOTAL                  185     55    70%

============================================================= 11 passed in 1.89s =============================================================
__________________________________________________________________ summary ___________________________________________________________________
ERROR:   py36: InvocationError for command /Users/tmac/github-projects/mfong/pypcap/.tox/py36/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/pypcap-1.3.0.zip (exited with code 1)
ERROR:   py37: InvocationError for command /Users/tmac/github-projects/mfong/pypcap/.tox/py37/bin/python -m pip install --no-deps -U .tox/.tmp/package/1/pypcap-1.3.0.zip (exited with code 1)
  py38: commands succeeded
  py39: commands succeeded
  py310: commands succeeded
mwfong-csl commented 2 years ago

@slhck What version of Python are you running? Per @tulay's post (above), you need Python 3.8 or newer to install pypcap on Mac OS >= 11. Also, I suggest not installing libpcap via brew, because libpcap.dylib is already distributed on those operating systems (albeit in a cached form) — this will also eliminate ambiguity in the header file location.

slhck commented 2 years ago

I tested with 3.9 and 3.10. Shouldn't it pick up the default libraries anyway? (I don't think the Homebrew ones would interfere here.)

mwfong-csl commented 2 years ago

It should have with Python >= 3.8 and Mac OS >= 11. I wonder that the difference is between your environment and Tulay's and koczanm's.

Per koczanm's previous comment (above), what is the result of your running

python39 -c "from ctypes.util import find_library; print(find_library('libpcap'))"

?

tulay commented 2 years ago

@slhck Try checking out @mwfong-csl's branch. There is no fix yet in the main repo which you seem to be trying to install from.

The original poster was also trying to install on python 2.7. The solution using ctypes will not work there. Unless 2.7 is a strict requirement, installing a python environment using conda or pyenv for 3.8+ is what I would do until there is a solution.

I just tested on python2.7 and it did install.

slhck commented 2 years ago

Ah, yes, checking out this branch makes it work: https://github.com/mwfong-csl/pypcap/tree/issue-104 — I had tried the master branch before.

So I guess this can be made a PR then?