search

borgbackup / borg

Deduplicating archiver with compression and authenticated encryption.
https://www.borgbackup.org/
Other
11.21k stars 743 forks source link

how to run self tests from pypi distfile #6386

Closed 0-wiz-0 closed 1 month ago

0-wiz-0 commented 2 years ago

I've updated the pkgsrc package, which is using the pypi distfile, to 1.2.0 and wanted to run the self tests to verify that the package works as expected.

I tried the following:

setup.py test

but this reported

...
Ran 0 tests in 0.000s

Then I tried using pytest, but got another error:

platform netbsd9 -- Python 3.10.2, pytest-7.0.1, pluggy-0.13.1                                                                                           
Tests enabled: BSD flags, root, symlinks, hardlinks, atime/mtime, modes                                                                                  
Tests disabled: fuse2, fuse3                                                
rootdir: /scratch/sysutils/py-borgbackup/work/borgbackup-1.2.0, configfile: setup.cfg
collected 0 items / 29 errors                                                                                                                                                                                                                                                                                     
======================================================================== ERRORS =========================================================================
____________________________________________________ ERROR collecting src/borg/testsuite/archive.py _____________________________________________________
import file mismatch:                                                                                                                                    
imported module 'borg.testsuite.archive' has this __file__ attribute:                                                                                    
  /usr/pkg/lib/python3.10/site-packages/borg/testsuite/archive.py                                                                                        
which is not the same as the test file we want to collect:                  
  /scratch/sysutils/py-borgbackup/work/borgbackup-1.2.0/src/borg/testsuite/archive.py                   
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
...
_____________________________________________________ ERROR collecting src/borg/testsuite/xattr.py ______________________________________________________import file mismatch:                                                                                                                                    imported module 'borg.testsuite.xattr' has this __file__ attribute:                                                                                        /usr/pkg/lib/python3.10/site-packages/borg/testsuite/xattr.py                                                                                          which is not the same as the test file we want to collect:                                                                                                 /scratch/sysutils/py-borgbackup/work/borgbackup-1.2.0/src/borg/testsuite/xattr.py                                                                      HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules                                                            ================================================================ short test summary info ================================================================ERROR src/borg/testsuite/archive.py                                                                                                                      ERROR src/borg/testsuite/archiver.py                                                                                                                     ERROR src/borg/testsuite/benchmark.py                                                                                                                    ERROR src/borg/testsuite/cache.py                                                                                                                        ERROR src/borg/testsuite/checksums.py                                                                                                                    ERROR src/borg/testsuite/chunker.py                                                                                                                      ERROR src/borg/testsuite/chunker_pytest.py                                                                                                               ERROR src/borg/testsuite/chunker_slow.py                                                                                                                 ERROR src/borg/testsuite/compress.py                                                                                                                     
ERROR src/borg/testsuite/crypto.py                                                                                                                       
ERROR src/borg/testsuite/efficient_collection_queue.py                                                                                                   
ERROR src/borg/testsuite/file_integrity.py                                                                                                               
ERROR src/borg/testsuite/hashindex.py                                                                                                                    
ERROR src/borg/testsuite/helpers.py                                         
ERROR src/borg/testsuite/item.py                                                                                                                         
ERROR src/borg/testsuite/key.py                                                                                                                          ERROR src/borg/testsuite/locking.py                                                                                                                      
ERROR src/borg/testsuite/logger.py                                                                                                                       
ERROR src/borg/testsuite/lrucache.py                                                                                                                     
ERROR src/borg/testsuite/nanorst.py                                                                                                                      
ERROR src/borg/testsuite/nonces.py                                                                                                                       
ERROR src/borg/testsuite/patterns.py                                                                                                                     
ERROR src/borg/testsuite/platform.py                                        
ERROR src/borg/testsuite/remote.py                                                                                                                       
ERROR src/borg/testsuite/repository.py                                                                                                                   
ERROR src/borg/testsuite/shellpattern.py                                                                                                                 
ERROR src/borg/testsuite/upgrader.py                                                                                                                     
ERROR src/borg/testsuite/version.py                                                                                                                      
ERROR src/borg/testsuite/xattr.py                                           
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 29 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================== 29 errors in 3.24s ===================================================================

What is the proper way to run the tests from the pypi distfile?

Environment: NetBSD 9.99.93/amd64, python 3.10.2, zstd 1.5.2, lz4 1.9.3, libb2 0.98.1, cython 0.29.27, msgpack 1.0.3, pytest 7.0.1.

ThomasWaldmann commented 2 years ago

The self-tests (a few quick tests) will run automatically each time you use borg.

We did not make any special provisions so that you can run the tests from the pypi package yet.

The tests need some addtl. packages, see requirements.d/development.txt and regularly run on github actions for each commit / pull request. Developers also can run them locally after doing a git checkout of the repo and setting up the environment as described in the install docs ("from source").

Also, when doing a release, I do vagrant-based platform testing, including on netbsd.

If you want to help with the netbsd situation, see #5922.

ThomasWaldmann commented 10 months ago

Trying to make this work for borg 1.4:

# optional: create and use a virtual env:
python3 -m venv env
. env/bin/activate

# install packages
pip install borgbackup-from-pypi.tgz
pip install pytest pytest-benchmark

# method A: use a pytest.ini

cat >pytest.ini <<<EOF
[pytest]
python_files = testsuite/*.py
markers = allow_cache_wipe
addopts = -v -rs --benchmark-skip
EOF

pytest --pyargs borg.testsuite

# method B: give the options via the cmdline (each time you invoke the tests):

pytest -v -rs --benchmark-skip -o 'python_files=testsuite/*.py' -o 'markers=allow_cache_wipe' --pyargs borg.testsuite
ThomasWaldmann commented 1 month ago

After #8452 is merged (master branch / borg2):

pytest -v -rs --benchmark-skip --pyargs borg.testsuite