webrecorder / warcio

Streaming WARC/ARC library for fast web archive IO
https://pypi.python.org/pypi/warcio
Apache License 2.0
383 stars 58 forks source link

get_test_file missing from the PyPI release #132

Open Apteryks opened 3 years ago

Apteryks commented 3 years ago

This leads to errors such as:

ImportError while importing test module '/tmp/guix-build-python-warcio-1.7.4.drv-0/warcio-1.7.4/test/test_writer.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
test/test_writer.py:11: in <module>
    from . import get_test_file
E   ImportError: attempted relative import with no known parent package
wumpus commented 3 years ago

Looks like test/init.py is explicitly excluded in setup.py? packages=find_packages(exclude=['test']) and as a result, nothing on pypi has it?

Apteryks commented 3 years ago

Hi; this is what the PyPI archive contains:

$ wget https://files.pythonhosted.org/packages/32/2c/c813f0576c8557ee50b9ecf54ba6c9a58b8cb3e7ca7109ca9fb37a56f4c3/warcio-1.7.4.tar.gz
--2021-08-26 13:16:34--  https://files.pythonhosted.org/packages/32/2c/c813f0576c8557ee50b9ecf54ba6c9a58b8cb3e7ca7109ca9fb37a56f4c3/warcio-1.7.4.tar.gz
Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.137.63, 2a04:4e42:20::319
Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.137.63|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 60515 (59K) [application/x-tar]
Saving to: ‘warcio-1.7.4.tar.gz’

warcio-1.7.4.tar.gz                                  100%[=====================================================================================================================>]  59.10K  --.-KB/s    in 0.02s   

2021-08-26 13:16:35 (2.35 MB/s) - ‘warcio-1.7.4.tar.gz’ saved [60515/60515]

maxim@hurd ~/Downloads$ tar tf warcio*
warcio-1.7.4/
warcio-1.7.4/PKG-INFO
warcio-1.7.4/LICENSE
warcio-1.7.4/test/
warcio-1.7.4/test/test_bufferedreaders.py
warcio-1.7.4/test/test_utils.py
warcio-1.7.4/test/test_statusandheaders.py
warcio-1.7.4/test/test_writer.py
warcio-1.7.4/test/test_capture_http_proxy.py
warcio-1.7.4/test/test_digestverifyingreader.py
warcio-1.7.4/test/test_archiveiterator.py
warcio-1.7.4/test/test_capture_http.py
warcio-1.7.4/test/test_check_digest_examples.py
warcio-1.7.4/test/test_limitreader.py
warcio-1.7.4/test/test_cli.py
warcio-1.7.4/warcio/
warcio-1.7.4/warcio/archiveiterator.py
warcio-1.7.4/warcio/capture_http.py
warcio-1.7.4/warcio/recompressor.py
warcio-1.7.4/warcio/__init__.py
warcio-1.7.4/warcio/warcwriter.py
warcio-1.7.4/warcio/extractor.py
warcio-1.7.4/warcio/limitreader.py
warcio-1.7.4/warcio/cli.py
warcio-1.7.4/warcio/utils.py
warcio-1.7.4/warcio/digestverifyingreader.py
warcio-1.7.4/warcio/indexer.py
warcio-1.7.4/warcio/recordloader.py
warcio-1.7.4/warcio/bufferedreaders.py
warcio-1.7.4/warcio/exceptions.py
warcio-1.7.4/warcio/recordbuilder.py
warcio-1.7.4/warcio/statusandheaders.py
warcio-1.7.4/warcio/timeutils.py
warcio-1.7.4/warcio/checker.py
warcio-1.7.4/MANIFEST.in
warcio-1.7.4/warcio.egg-info/
warcio-1.7.4/warcio.egg-info/PKG-INFO
warcio-1.7.4/warcio.egg-info/zip-safe
warcio-1.7.4/warcio.egg-info/SOURCES.txt
warcio-1.7.4/warcio.egg-info/entry_points.txt
warcio-1.7.4/warcio.egg-info/requires.txt
warcio-1.7.4/warcio.egg-info/top_level.txt
warcio-1.7.4/warcio.egg-info/dependency_links.txt
warcio-1.7.4/NOTICE
warcio-1.7.4/setup.py
warcio-1.7.4/setup.cfg
warcio-1.7.4/README.rst
warcio-1.7.4/CHANGELIST.rst
wumpus commented 3 years ago

I agree, I already looked at that. The point about not shipping test is that it's namespace pollution, it's not below warcio. All of the packages I've ever packaged didn't ship the tests, for this reason. Did I miss a new trend in packaging python?

Apteryks commented 3 years ago

I think the issue I saw was that the tests appear to be partially included in the release archive; if the tests are explicitly excluded, that's OK, but they should fully be (i.e., there shouldn't be any "test" directory remaining in the root directory).