Closed kloczek closed 6 months ago
In the same build env 7.0.3 passes pytest without any issues.
I'll explain this a bit more in depth here, that way if/when other tickets come in with similar issues, I can reference this ticket for the explanation once.
There is no functional change between 7.0.3
and 7.0.4
, just the ability to set NGROK_AUTHTOKEN
from the environment. Everything else in the diff is build improvements.
So, what did change between 7.0.3
and 7.0.4
? It's environmental from the ngrok
service, not the pyngrok
library. Your test failures are all failing (and skipped) for the same reason and telling you why: pyngrok.exception.PyngrokNgrokError: The ngrok process errored on start: authentication failed: Usage of ngrok requires a verified account and authtoken.\n\nSign up for an account: https://dashboard.ngrok.com/signup\nInstall your authtoken: https://dashboard.ngrok.com/get-started/your-authtoken\r\n\r\nERR_NGROK_4018\r\n.
Around December 20th, 2023 (or therabouts), ngrok
stopped allowing anonymous usage to its service, and requires all usage of its APIs and binaries to use an auth token. Though your 7.0.3
build previously worked, if you reran it after this date, it would fail with the same error as you're seeing now running against 7.0.4
. This is actually why 7.0.4
was released, so that developers could easily set the auth token from an environment variable without having to change anything in their code.
The solution: set NGROK_AUTHTOKEN
in your environment with your auth token found at https://dashboard.ngrok.com/get-started/your-authtoken, then rerun your build.
Issue is that units which requires access to public network like in many other modules should be possible to skip over network
pytest mark.
Most of the distributions build infras are intentionally cut off from access to the public network.
That is a problem with the nature of pyngrok
's test suite, that to validate there are no regressions, we have to connect to the ngrok
service, and we have to use an auth token. This same "issue" would have existed in all versions of pyngrok
before 7.0.4
too though, the fact is it's an integration test suite and not a unit test suite, since it's a library that's dependent on an external, internet-connected service.
I've considered making a subset smoke suite of true unit tests that mock out the third party responses, this would also resolve the issue we see with pull request builds not working since GitHub doesn't share the auth token secret to pull requests, but the fact is there hasn't been a huge need for it (compared to the work it'd be to do it) to this point. I'd of course consider a pull request that attempted this, but like I said, I know it wouldn't be a small effort to make that change.
I fully understand situation however part of the units do not need network access and IMO it would be nice to to have possibility to exclude automatically such units depends on configuration of build system. Simple sometimes "something it is better than nothing".
Can you try running your build against the develop
branch? Since ngrok
made their change last week, all the new tests that now also require an auth token are skipped. If that works for you, I can merge it in later this week and cut a 7.0.5
that I think will accomplish what you want.
Result against 7.0.4 + three commits from branch
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pyngrok-7.0.4-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pyngrok-7.0.4-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.18, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/tkloczko/rpmbuild/BUILD/pyngrok-7.0.4
collected 72 items
tests/test_conf.py s [ 1%]
tests/test_installer.py s.s..ss. [ 12%]
tests/test_ngrok.py ssss.sssssssssssssssssss....sssss.sss...sss.s.. [ 77%]
tests/test_process.py s.sssssss...ssss [100%]
================================================================================== short test summary info ==================================================================================
SKIPPED [1] tests/test_conf.py:13: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_installer.py:68: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_installer.py:47: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_installer.py:17: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_installer.py:32: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:259: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:297: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:315: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:273: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:334: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:392: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:409: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:681: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:129: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:149: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:139: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:189: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:450: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:102: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:40: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:71: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:474: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:199: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:220: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:491: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:112: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:239: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:706: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:507: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:617: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:428: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:349: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:371: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:179: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:159: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:169: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:729: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:527: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:567: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_ngrok.py:765: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_process.py:299: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_process.py:170: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_process.py:228: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_process.py:246: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_process.py:316: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_process.py:117: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_process.py:141: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_process.py:264: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_process.py:37: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_process.py:77: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_process.py:329: NGROK_AUTHTOKEN environment variable not set
SKIPPED [1] tests/test_process.py:21: NGROK_AUTHTOKEN environment variable not set
============================================================================== 20 passed, 52 skipped in 6.30s ===============================================================================
I think that it is enough 👍 😄
Thx
I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.
python3 -sBm build -w --no-isolation
build
with--no-isolation
I'm using during all processes only locally installed modulescut off from access to the public network
(pytest is executed with-m "not network"
)List of installed modules in build env:
```console Package Version ----------------------------- ------- alabaster 0.7.13 Babel 2.14.0 build 1.0.3 charset-normalizer 3.3.2 cppclean 0.13 distro 1.8.0 dnf 4.18.2 docutils 0.20.1 exceptiongroup 1.1.3 gpg 1.23.2 idna 3.6 imagesize 1.4.1 importlib-metadata 7.0.1 iniconfig 2.0.0 installer 0.7.0 Jinja2 3.1.2 libdnf 0.72.0 MarkupSafe 2.1.3 packaging 23.2 pluggy 1.3.0 psutil 5.9.5 Pygments 2.17.2 pyproject_hooks 1.0.0 pytest 7.4.3 python-dateutil 2.8.2 pytz 2023.3 PyYAML 6.0.1 requests 2.31.0 setuptools 69.0.3 six 1.16.0 snowballstemmer 2.2.0 Sphinx 7.1.2 sphinx-autodoc-typehints 1.25.2 sphinx-notfound-page 1.0.0 sphinxcontrib-applehelp 1.0.4 sphinxcontrib-devhelp 1.0.5 sphinxcontrib-htmlhelp 2.0.4 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.9 tomli 2.0.1 urllib3 1.26.18 wheel 0.42.0 zipp 3.17.0 ```