Closed kloczek closed 6 months ago
musicinmybrain
commented
7 months ago I see the same test failure while working on updating the python-rope package in Fedora Linux. It looks like there were several test failures in the previous version, but this is the only one remaining in 1.12.0. I haven’t made a serious effort to find the cause of the failure, but I’m also happy to do any investigation that might help here upon request.
lieryan
commented
6 months ago This should be fixed now in #755.
kloczek
commented
6 months ago Just tested 0.23.0 + all commits from master (up to 4deab03c) and pytest still fails
kloczek
commented
6 months ago One sec I see that now test suite wants pip ..
kloczek
commented
6 months ago Hmm looks like pip is used to install external_fixturepkg module.
If that module needs be installed to test something please use pip inside venv to do that instead polluting list of installed modules.
Additionally pytest shows some DeprecationWarning warnings
kloczek
commented
6 months ago I'm going to add in my package build procedure (temporary) add to --deselect list those 3 failing units to have working build procedure.
lieryan
commented
6 months ago Thanks for the feedback @kloczek. Hmm... I think that would mean we'd need to create a venv. IIRC, venv should've also installed pip into the venv, but I'm not quite sure if it requires network to do that.
Additionally pytest shows some DeprecationWarning warnings
The DeprecationWarnings are fine. They are all issued by rope for the benefit of editor integration authors, we have nothing to fix here on rope's side. Maybe they can be suppressed during tests though since they're just noise in the tests.
kloczek
commented
6 months ago I'm building all my packages in dedicated build env in which are installed ONLY exact build dependencies so effectively already it is like venv. Is it anything wrong with list of modules with versions which I've dropped opening this ticket? 🤔
lieryan
commented
6 months ago The tests needs to install external_fixturepkg, because some of the tests needs to have a package with a predictable structure to test that autoimport is indexing the package correctly. The test installs the external_fixturepkg package from a .whl file that's crafted specifically for testing autoimport and is included prepackaged in rope's source archive.
It could probably be theoretically possible to setup and install the test package without using pip, but I'm not familiar enough with how package installation process or venv works to trust that I'd have set things up properly, especially as venv have slightly different structure in different OSes.
kloczek
commented
6 months ago Moment so you want to say that rope can alter its functionality by downloading from public network some random .whl archives and you need to test that? 🤔
If that is true PLEASE stop that and please FLAG that some modules are missing because it is nothing more than asking for security related issue.
lieryan
commented
6 months ago No, that wasn't what I was saying, rope does not need to install anything at runtime.
It's ropetest, our test suite, that need to install a package because it need to simulate what the user is doing, which is installing a package into their project's environment. Installing a package is part of a test's setting up a fixture, not the system under test (SUT). And it's not random .whl, this wheel is built specifically for ropetest and is included in the test suite itself, specifically because I don't want it to download random .whls off the internet even if it's just for running tests.
kloczek
commented
6 months ago It's ropetest, our test suite, that need to install a package because it need to simulate what the user is doing, which is installing a package into their project's environment. Installing a package is part of a test's setting up a fixture, not the system under test (SUT).
Why test suite needs to simulate user? 🤔 Test suite should test module CODE.
And it's not random .whl, this wheel is built specifically for ropetest and is included in the test suite itself, specifically because I don't want it to download random .whls off the internet even if it's just for running tests.
Whatever is the reason downloading and installing anything during the test suite execution is bad.
Did you know that most of the distros production build infras are intentionally cut off from access to the public network? 🤔
Please add that fact to your testing methodology.
Many python modules test suite already support pytest mark network to allow easy skip all units requiring access to the public network.
lieryan
commented
6 months ago Did you know that most of the distros production build infras are intentionally cut off from access to the public network? 🤔
And that is perfectly fine. Again, as I keep saying, the test suite is not downloading anything; installing using a package manager does not necessarily entail downloading off public network. The .whl package that we need is included as part of the test suite, pip do not require network access to install a wheel that has no dependency. All that is needed is installing a wheel file that is included in the test suite, in an environment that is controlled by the test suite.
Rope is a developer tooling library, it analyses the packages you have installed into your python environment to provide the autoimport/autocompletion services. To a program like rope, the content of the project's venv is input files, not code dependencies. That's the important distinction that you seem to be missing here.
We need to create a directory containing files that looks like a venv folder that users would have set up for their Python project because that's the input file for a developer tool. I'm not quite sure what else we can do here short of just poorly reimplementing what venv and pip does as test setup code.
python3 -sBm build -w --no-isolationbuildwith--no-isolationI'm using during all processes only locally installed modulescut off from access to the public network(pytest is executed with-m "not network")Here is pytest output:
```console + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rope-1.12.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rope-1.12.0-2.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' ==================================================================================== test session starts ==================================================================================== platform linux -- Python 3.8.18, pytest-7.4.4, pluggy-1.3.0 rootdir: /home/tkloczko/rpmbuild/BUILD/rope-1.12.0 configfile: pyproject.toml plugins: subprocess-1.5.0 collected 2072 items ropetest/advanced_oi_test.py ............................................................ [ 2%] ropetest/builtinstest.py ............................................................... [ 5%] ropetest/codeanalyzetest.py ................................................................................................................................................... [ 13%] ropetest/doatest.py .... [ 13%] ropetest/historytest.py ........................................ [ 15%] ropetest/objectdbtest.py ................... [ 16%] ropetest/objectinfertest.py ................................. [ 17%] ropetest/projecttest.py ........................................................................................................................ [ 23%] ropetest/pycoretest.py ................................................................................................................................ [ 29%] ropetest/pyscopestest.py .............................................. [ 31%] ropetest/reprtest.py ................ [ 32%] ropetest/runmodtest.py ......... [ 33%] ropetest/serializer_test.py ............................................................ [ 35%] ropetest/simplifytest.py ............... [ 36%] ropetest/type_hinting_test.py ..............................x.................x.................x.................x.......... [ 41%] ropetest/versioningtest.py ..... [ 41%] ropetest/contrib/autoimporttest.py .................F........ [ 42%] ropetest/contrib/changestacktest.py . [ 42%] ropetest/contrib/codeassisttest.py .................................................................................................................................................. [ 49%] ........................ [ 51%] ropetest/contrib/finderrorstest.py ..... [ 51%] ropetest/contrib/findittest.py ........... [ 51%] ropetest/contrib/fixmodnamestest.py .... [ 51%] ropetest/contrib/generatetest.py .............................. [ 53%] ropetest/contrib/autoimport/autoimporttest.py ................ [ 54%] ropetest/contrib/autoimport/modeltest.py ........... [ 54%] ropetest/contrib/autoimport/parsetest.py ... [ 54%] ropetest/contrib/autoimport/utilstest.py ........... [ 55%] ropetest/refactor/__init__.py ...................................................................... [ 58%] ropetest/refactor/change_signature_test.py ........................................ [ 60%] ropetest/refactor/extracttest.py .............................................s..........sss....x......................s............................................................. [ 67%] ...................... [ 68%] ropetest/refactor/importutilstest.py ............................................................................................................................................... [ 75%] ropetest/refactor/inlinetest.py ............................................................................................... [ 80%] ropetest/refactor/movetest.py .......................................................................... [ 83%] ropetest/refactor/multiprojecttest.py .... [ 84%] ropetest/refactor/patchedasttest.py ..........................s..s.s.........................................................ssssssssss............................s.s.s..s.......... [ 91%] . [ 91%] ropetest/refactor/renametest.py ......................................................................................... [ 95%] ropetest/refactor/restructuretest.py ....................... [ 96%] ropetest/refactor/similarfindertest.py ..................................... [ 98%] ropetest/refactor/suitestest.py .......s............. [ 99%] ropetest/refactor/usefunctiontest.py ............ [100%] ========================================================================================= FAILURES ========================================================================================== ___________________________________________________________________________ AutoImportTest.test_search_submodule ____________________________________________________________________________ self =That unit has been failing with prev version as well.
List of installed modules in build env:
```console Package Version ----------------------------- ------- alabaster 0.7.16 Babel 2.14.0 build 1.0.3 charset-normalizer 3.3.2 cppclean 0.13 distro 1.9.0 dnf 4.18.2 docutils 0.20.1 exceptiongroup 1.1.3 gpg 1.23.2 idna 3.6 imagesize 1.4.1 importlib-metadata 7.0.1 iniconfig 2.0.0 installer 0.7.0 Jinja2 3.1.3 libdnf 0.72.0 MarkupSafe 2.1.3 nodeenv 1.8.0 packaging 23.2 platformdirs 4.1.0 pluggy 1.3.0 Pygments 2.17.2 pyproject_hooks 1.0.0 pytest 7.4.4 pytest-subprocess 1.5.0 python-dateutil 2.8.2 pytoolconfig 1.3.1 pytz 2023.3 PyYAML 6.0.1 requests 2.31.0 setuptools 69.0.3 six 1.16.0 snowballstemmer 2.2.0 Sphinx 7.1.2 sphinx-autodoc-typehints 1.25.2 sphinx-rtd-theme 2.0.0 sphinxcontrib-applehelp 1.0.4 sphinxcontrib-devhelp 1.0.5 sphinxcontrib-htmlhelp 2.0.5 sphinxcontrib-jquery 4.1 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.7 sphinxcontrib-serializinghtml 1.1.10 tabulate 0.9.0 tomli 2.0.1 urllib3 1.26.18 wheel 0.42.0 zipp 3.17.0 ```Please let me know if you need more details or want me to perform some diagnostics.