Open geofire opened 3 weeks ago
Based on the provided information it seems like importing the Library is unreasonably expensive
Please validate if importing lazyly removes the stall
Hi @RonnyPfannschmidt,
Lazy loading certainly appears to bypass the stall (with mocking code commented out in the test):
# app.py
import os
# from arcgis import GIS # Moved from here to connect()
def connect():
from arcgis import GIS # Lazily load GIS from arcgis
portal = GIS(url=connection_url(domain, context), username=os.getenv('USERNAME'), username=os.getenv('PASSWORD'))
return portal
def connection_url(domain, context):
if domain == 'arcgis.com'
return None # API defaults to arcgis.com if None used as parameter
else:
return(f"https://{domain}/{context}")
pytest-richtrace
doesn't show pytest
traversing into arcgis
.
Hi all!
I was tossing up whether this should be a bug report or a feature request, as I wasn't able to work out whether the following is expected behaviour in the documention. It took a good day of solid troubleshooting to figure this one out, which I have found a workaround, so it's not a showstopper though it was difficult and very confusing to troubleshoot.
What's the problem?
pytest
will, perhaps by design?, scan through and 'collect' (some, all?) third-party libraries used inx
function, whenx
function iself is imported into a test to run.This gives the impression that
pytest
:Current behaviour (as of
pytest 8.3.2
)Note: I've used the awesome
pytest-richtrace
library (in--verbose
mode) to help me figure this issue out, as there didn't appear to be a similar function inpytest
to makepytest
s collection activities verbose.The library I can reliably reproduce this issue with is
arcgis
. https://pypi.org/project/arcgis/arcgis
is Esri's ArcGIS API for Python, allowing Python code to interact with Esri's Enterprise and Online geospatial systems without needing to write a mess of boilerplate REST API code.arcgis
is a package I don't maintain, and don't need to test directly.The same behaviour is present in both PyCharm and by manually invoking
pytest
withpython -m
via command line.Example
Tests
Note that:
connect()
is not called,connect()
,connect()
isn't imported intotest_app.py
,connection_url()
function does not callconnect()
, and therefore callarcgis.GIS
, andtest_app.py
essentially does nothing other than import connection_url from app.pyI have also explicitly excluded
venv
andsite-packages
as directories in pytest.ini.The code above, as it is written right now, will see
pytest
traverse thearcgis
package within the virtual environment that runs this code. According topytest-richtrace
,pytest
doesn't appear to collect anything in that package or the venv directory.pytest
seems to ignore theos
package.Using
pytest-richtrace
I saw the following behaviour:Simply commenting out the import line in
test_app.py
---- prevents
pytest
traversing intoarcgis
to collect. All other tests complete pretty much instantaneously.I haven't checked whether
arcgis
has any tests though the collection process certainly doesn't pick any up.Describe the solution you'd like
pytest
to exclude certain libraries from collection. (excluding directories doesn't do this)--verbose
flag, so that it's much easier to troubleshoot collection issues:--durations=0
flag is calledWorkaround solution
Using
MagicMock
inunittest.mock
allowspytest
to traverse intoapp.py
without also traversing intoarcgis
, stopping the stalling issue without having to comment out code or refactor unnecessarily:Many thanks!