tests: $HOME not set?

[boyanpenkov]

After https://github.com/perrette/papers/pull/53 -- which is a good idea! -- I see:

(python311) → master Work/papers tox -e py311                                                    8:07:09
GLOB sdist-make: /home/boyan/boyanshouse/Vazhno/Work/papers/setup.py
py311 create: /home/boyan/boyanshouse/Vazhno/Work/papers/.tox/py311
py311 installdeps: bibtexparser, scholarly, crossrefapi, rapidfuzz, unidecode, normality, pytest, pytest-cov
py311 inst: /home/boyan/boyanshouse/Vazhno/Work/papers/.tox/.tmp/package/1/papers-cli-2.3.dev130+gfd9291b.zip
py311 installed: alabaster==0.7.13,anyio==3.6.2,arrow==1.2.3,async-generator==1.10,attrs==23.1.0,Babel==2.12.1,banal==1.0.6,beautifulsoup4==4.12.2,bibtexparser==1.4.0,certifi==2022.12.7,chardet==5.1.0,charset-normalizer==3.1.0,coverage==7.2.4,crossrefapi==1.5.0,Deprecated==1.2.13,docutils==0.18.1,exceptiongroup==1.1.1,fake-useragent==1.1.3,free-proxy==1.1.1,h11==0.14.0,httpcore==0.17.0,httpx==0.24.0,idna==3.4,imagesize==1.4.1,iniconfig==2.0.0,Jinja2==3.1.2,lxml==4.9.2,MarkupSafe==2.1.2,normality==2.4.0,outcome==1.2.0,packaging==23.1,papers-cli @ file:///home/boyan/boyanshouse/Vazhno/Work/papers/.tox/.tmp/package/1/papers-cli-2.3.dev130%2Bgfd9291b.zip,pluggy==1.0.0,Pygments==2.15.1,pyparsing==3.0.9,PySocks==1.7.1,pytest==7.3.1,pytest-cov==4.0.0,python-dateutil==2.8.2,python-dotenv==1.0.0,rapidfuzz==3.0.0,requests==2.29.0,scholarly==1.7.11,selenium==4.9.0,six==1.16.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.4.1,Sphinx==6.2.1,sphinx-rtd-theme==1.2.0,sphinxcontrib-applehelp==1.0.4,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.1,sphinxcontrib-jquery==4.1,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,text-unidecode==1.3,trio==0.22.0,trio-websocket==0.10.2,typing_extensions==4.5.0,Unidecode==1.3.6,urllib3==1.26.15,wrapt==1.15.0,wsproto==1.2.0
py311 run-test-pre: PYTHONHASHSEED='2774689394'
py311 run-test: commands[0] | pytest --cov=papers --cov-append --cov-report=term-missing -xv
========================================== test session starts ==========================================
platform linux -- Python 3.11.3, pytest-7.3.1, pluggy-1.0.0 -- /home/boyan/boyanshouse/Vazhno/Work/papers/.tox/py311/bin/python
cachedir: .tox/py311/.pytest_cache
rootdir: /home/boyan/boyanshouse/Vazhno/Work/papers
plugins: cov-4.0.0, anyio-3.6.2
collected 0 items / 1 error                                                                             

================================================ ERRORS =================================================
__________________________________ ERROR collecting tests/test_add.py ___________________________________
../../../miniconda3/envs/python311/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1206: in _gcd_import
    ???
<frozen importlib._bootstrap>:1178: in _find_and_load
    ???
<frozen importlib._bootstrap>:1128: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1206: in _gcd_import
    ???
<frozen importlib._bootstrap>:1178: in _find_and_load
    ???
<frozen importlib._bootstrap>:1149: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:690: in _load_unlocked
    ???
<frozen importlib._bootstrap_external>:940: in exec_module
    ???
<frozen importlib._bootstrap>:241: in _call_with_frames_removed
    ???
tests/__init__.py:3: in <module>
    sp.check_call('git config --list | grep user.name || git config --global user.name "Papers Tests"', shell=True)
../../../miniconda3/envs/python311/lib/python3.11/subprocess.py:413: in check_call
    raise CalledProcessError(retcode, cmd)
E   subprocess.CalledProcessError: Command 'git config --list | grep user.name || git config --global user.name "Papers Tests"' returned non-zero exit status 128.
-------------------------------------------- Captured stderr --------------------------------------------
fatal: $HOME not set
======================================== short test summary info ========================================
ERROR tests/test_add.py - subprocess.CalledProcessError: Command 'git config --list | grep user.name || git config --global us...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
=========================================== 1 error in 0.22s ============================================
ERROR: InvocationError for command /home/boyan/boyanshouse/Vazhno/Work/papers/.tox/py311/bin/pytest --cov=papers --cov-append --cov-report=term-missing -xv (exited with code 1)
________________________________________________ summary ________________________________________________
ERROR:   py311: commands failed

Not sure what the thing is here...

[boyanpenkov]

There was some suspicion about my conda env, since papers --version and papers -h didn't work, but I reinstalled it (all from conda-forge), the papers commands work, but the problem persists.

[boyanpenkov]

OK, this is tox being too smart for it's own good, since if I just run the test string pytest --cov=papers --cov-append --cov-report=term-missing -xv I get 100% pass on all 145 tests.

Looking at this now..

[boyanpenkov]

Based on https://github.com/pre-commit/pre-commit-hooks/issues/402 I think we may need a HOME = {homedir} somewhere, but since we don't have a tox.ini not sure where...

[perrette]

• At the moment $HOME and other environment variables are used in config.py towards the beginning
• tox.ini can be found inside pyproject.toml

[boyanpenkov]

OK, so, trying this:

-- I confirm I have HOME = os.environ.get('HOME',os.path.expanduser('~')) in papers/config.py and that this executes cleanly on my machine:

Python 3.11.3 (main, Apr 19 2023, 23:54:32) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> HOME = os.environ.get('HOME',os.path.expanduser('~'))
>>> print(HOME)
/home/boyan
>>> 

-- adding the homedir explicitly to pyproject.toml, which should be a global tox substitution:

[tool.tox]
# Tox (https://tox.readthedocs.io/) is a tool for running tests                                                                                                                                       
# in multiple virtualenvs. This configuration file will run the                                                                                                                                       
# test suite on all supported python versions. To use it, "pip install tox"                                                                                                                           
# and then run "tox" from this directory.                                                                                                                                                             
legacy_tox_ini = "HOME={homedir}"
[tox]
envlist = clean, py38, py39, py310, py311, report

fails with a tox "internal error"

[perrette]

I'm not sure what's the correct way of setting the home directory, but your snippet has a syntax error. legacy_tox_ini opens a triple quote and it's closed further down. Should be like that:

legacy_tox_ini = """
HOME={homedir}
...
"""

See syntax highlighting here: https://github.com/perrette/papers/blob/master/pyproject.toml

[boyanpenkov]

Yes, you're totally right here -- I'm looking at this in https://github.com/perrette/papers/pull/59 and now have the $HOME set, I think correctly, but now the synthetic object that exists for backup seens the wrong directory:

tests/test_install.py::TestDefaultLocal2::test_install FAILED                                                                                                                                  [ 82%]

============================================================================================== FAILURES ==============================================================================================
___________________________________________________________________________________ TestDefaultLocal2.test_install ___________________________________________________________________________________
tests/common.py:224: in setUp
    self.config = Config.load(CONFIG_FILE)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

cls = <class 'papers.config.Config'>, file = '{homedir}/.local/share/config.json'

    @classmethod
    def load(cls, file):
>       js = json.load(open(file))
E       FileNotFoundError: [Errno 2] No such file or directory: '{homedir}/.local/share/config.json'

papers/config.py:121: FileNotFoundError
---------------------------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------------------------
papers configuration (global)
version 2.3.dev134+g88d6f6c.d20230510
* configuration file: {homedir}/.local/share/config.json
* cache directory:    {homedir}/.cache/papers
* absolute paths:     True
* git-tracked:        False
* editor:             None
* files directory:    filesxyz (empty)
* bibtex:             papersxyz.bib (empty)
----------------------------------------------------------------------------------------- Captured log call ------------------------------------------------------------------------------------------
INFO     papers:__main__.py:285 create empty bibliography database: papersxyz.bib
INFO     papers:__main__.py:292 create empty files directory: filesxyz
INFO     papers:__main__.py:327 save config file: {homedir}/.local/share/config.json
========================================================================================== warnings summary ==========================================================================================

[boyanpenkov]

Actually, for this thing -- is there a way to try with tox 4.4.4 someway? https://github.com/tox-dev/tox/issues/2702 looking now...

[boyanpenkov]

yep, confirming pip install tox==4.5.1 does not fix anything; bummer...