Github Actions and Import Errors

belthaZornv commented 4 years ago

I'm currently setting up pyre to run in a github action - I've been facing issues with:

Could not find a module corresponding to import sentry_sdk.

Among other errors - I've edited the .pyre_configuration to try and align with github actions (not really knowing where the env is set up):

{
  "binary": "/opt/hostedtoolcache/Python/3.7.8/x64/bin/pyre.bin",
  "source_directories": [
    "./sike/"
  ],
  "taint_models_path": "/opt/hostedtoolcache/Python/3.7.8/x64/lib/pyre_check/taint/",
  "search_path": ["/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages"],
  "typeshed": "/opt/hostedtoolcache/Python/3.7.8/x64/lib/pyre_check/typeshed/"
}

My folder structure is basically the root folder contains the manage.py and some configurations (django app), then I have a sike folder with the apps, config etc.

I've also tried using:

{
  "binary": "/opt/hostedtoolcache/Python/3.7.8/x64/bin/pyre.bin",
  "source_directories": [
    "."
  ],
  "taint_models_path": "/opt/hostedtoolcache/Python/3.7.8/x64/lib/pyre_check/taint/",
  "search_path": ["/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/"],
  "typeshed": "/opt/hostedtoolcache/Python/3.7.8/x64/lib/pyre_check/typeshed/"
}

Could you kindly help me set this up?

gbleaney commented 4 years ago

@belthaZornv could you verify that you see a sentry_sdk folder in your /opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages? For some reason, Pyre can't find sentry_sdk in any of the folders in your search_path, and the simplest answer might just be that it's installed elsewhere.

belthaZornv commented 4 years ago

It's running in github actions - I'll try to run a find command to make sure where sentry_sdk is placed. Thanks for now @gbleaney , appreciate it.

belthaZornv commented 4 years ago

I've managed to find where the site-packages are stored / cached on github actions:


  "binary": "/opt/hostedtoolcache/Python/3.7.8/x64/bin/pyre.bin",
  "source_directories": [
    "."
  ],
  "taint_models_path": "/opt/hostedtoolcache/Python/3.7.8/x64/lib/pyre_check/taint/",
  "search_path": [
    "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/",
    "/opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/"
  ],
  "typeshed": "/opt/hostedtoolcache/Python/3.7.8/x64/lib/pyre_check/typeshed/"
}

Although errors are still showing, and I'm 100% certain the packages are there. - am I missing something?

gbleaney commented 4 years ago

@belthaZornv is the error still the same message?

Can you try including the import line that is triggering this error message?

Also, can you include the output of ls -l /opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/ and /opt/hostedtoolcache/Python/3.7.8/x64/lib/python3.7/site-packages/sentry_sdk?

belthaZornv commented 4 years ago

@gbleaney yes, same message dude.

The output is:

-rwxr-xr-x+  1 runner docker      64079826 Aug 13 06:58 3220aef3144e83d3e26a__mypyc.cpython-37m-x86_64-linux-gnu.so
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 APScheduler-3.6.3.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 Django-3.0.8.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 Faker-4.1.1.dist-info
drwxrwxrwx+ 12 runner docker          4096 Aug 13 06:58 IPython
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 PyYAML-5.3.1-py3.7.egg-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 Pygments-2.6.1.dist-info
-rwxrwxrwx   1 runner runneradmin      119 Aug  6 20:24 README.txt
drwxrwxrwx+  2 runner runneradmin     4096 Aug 13 06:59 __pycache__
drwxrwxrwx+  8 runner docker          4096 Aug 13 06:59 _pytest
drwxrwxrwx+  7 runner docker          4096 Aug 13 06:58 apscheduler
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 asgiref
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 asgiref-3.2.10.dist-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 attr
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 attrs-19.3.0.dist-info
drwxrwxrwx+  5 runner docker          4096 Aug 13 06:58 audit_log
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 backcall
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 backcall-0.2.0.dist-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 certifi
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 certifi-2020.6.20.dist-info
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:58 chardet
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 chardet-3.0.4.dist-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 colorama
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 colorama-0.3.7.dist-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 corsheaders
drwxrwxrwx+  5 runner docker          4096 Aug 13 06:58 coverage
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 coverage-5.2.1.dist-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 creditcards
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:59 curl
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 dataclasses-0.6.dist-info
-rw-rw-rw-+  1 runner docker         45208 Aug 13 06:58 dataclasses.py
drwxrwxrwx+  6 runner docker          4096 Aug 13 06:58 dateutil
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 decorator-4.4.2.dist-info
-rw-rw-rw-+  1 runner docker         17222 Aug 13 06:58 decorator.py
drwxrwxrwx+ 19 runner docker          4096 Aug 13 06:58 django
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:58 django_apscheduler
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 django_apscheduler-0.4.1-py3.7.egg-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 django_audit_log-0.7.0.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 django_cors_headers-3.4.0.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 django_credit_cards-0.4.1.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 django_environ-0.4.5.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 django_filter-2.3.0.dist-info
drwxrwxrwx+  6 runner docker          4096 Aug 13 06:58 django_filters
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 django_grappelli-2.14.2-py3.7.egg-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 djangorestframework-3.11.0.dist-info
-rwxrwxrwx   1 runner runneradmin      126 Aug  6 20:24 easy_install.py
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 environ
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 factory
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 factory_boy-2.12.0.dist-info
drwxrwxrwx+  6 runner docker          4096 Aug 13 06:58 faker
drwxrwxrwx+  8 runner docker          4096 Aug 13 06:58 flake8
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 flake8-3.8.3.dist-info
drwxrwxrwx+ 10 runner docker          4096 Aug 13 06:58 future
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 future-0.18.2-py3.7.egg-info
drwxrwxrwx+  9 runner docker          4096 Aug 13 06:58 grappelli
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 idna
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 idna-2.10.dist-info
drwxrwxrwx+  5 runner docker          4096 Aug 13 06:58 importlib_metadata
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 importlib_metadata-1.7.0.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 iniconfig-1.0.0-py3.7.egg-info
-rw-r--r--+  1 runner docker          5208 Sep 23  2016 iniconfig.py
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 ipdb
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 ipdb-0.13.3-py3.7.egg-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 ipython-7.17.0.dist-info
drwxrwxrwx+  5 runner docker          4096 Aug 13 06:58 ipython_genutils
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 ipython_genutils-0.2.0.dist-info
drwxrwxrwx+  7 runner docker          4096 Aug 13 06:58 jedi
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 jedi-0.17.2.dist-info
drwxrwxrwx+ 12 runner docker          4096 Aug 13 06:58 libcst
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 libcst-0.3.9.dist-info
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:58 libfuturize
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:58 libpasteurize
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 mccabe-0.6.1.dist-info
-rw-rw-rw-+  1 runner docker         10693 Aug 13 06:58 mccabe.py
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 more_itertools
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 more_itertools-8.4.0.dist-info
drwxrwxrwx+  9 runner docker         12288 Aug 13 06:58 mypy
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 mypy-0.782.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 mypy_extensions-0.4.3.dist-info
-rw-rw-rw-+  1 runner docker          5078 Aug 13 06:58 mypy_extensions.py
drwxrwxrwx+ 13 runner docker          4096 Aug 13 06:58 mypyc
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 packaging
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 packaging-20.4.dist-info
drwxrwxrwx+  5 runner docker          4096 Aug 13 06:58 parso
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 parso-0.7.1.dist-info
drwxrwxrwx+  7 runner docker          4096 Aug 13 06:58 past
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 pexpect
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 pexpect-4.8.0.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 pickleshare-0.7.5.dist-info
-rw-rw-rw-+  1 runner docker          9942 Aug 13 06:58 pickleshare.py
drwxrwxrwx+  5 runner runneradmin     4096 Aug 13 06:58 pip
drwxrwxrwx+  2 runner runneradmin     4096 Aug  6 20:24 pip-20.1.1.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 pip-20.2.2.dist-info
drwxrwxrwx+  5 runner runneradmin     4096 Aug  6 20:24 pkg_resources
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 pluggy
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 pluggy-0.13.1.dist-info
drwxrwxrwx+ 18 runner docker          4096 Aug 13 06:58 prompt_toolkit
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 prompt_toolkit-3.0.5.dist-info
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:58 psutil
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 psutil-5.7.2-py3.7.egg-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 psycopg2
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 psycopg2-2.8.5-py3.7.egg-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 ptyprocess
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 ptyprocess-0.6.0.dist-info
drwxrwxrwx+  9 runner docker          4096 Aug 13 06:58 py
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 py-1.9.0.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 pycodestyle-2.6.0.dist-info
-rw-rw-rw-+  1 runner docker        103376 Aug 13 06:58 pycodestyle.py
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 pycurl-7.43.0.5-py3.7.egg-info
-rwxr-xr-x+  1 runner docker        632552 Aug 13 06:59 pycurl.cpython-37m-x86_64-linux-gnu.so
drwxrwxrwx+  5 runner docker          4096 Aug 13 06:58 pyflakes
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 pyflakes-2.2.0.dist-info
drwxrwxrwx+  7 runner docker          4096 Aug 13 06:58 pygments
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 pyparsing-2.4.7.dist-info
-rw-rw-rw-+  1 runner docker        273365 Aug 13 06:58 pyparsing.py
drwxrwxrwx+  5 runner docker          4096 Aug 13 06:59 pyre_check
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 pyre_check-0.0.52.dist-info
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:59 pyre_extensions
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 pyre_extensions-0.0.18.dist-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:59 pytest
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 pytest-6.0.1.dist-info
-rw-rw-rw-+  1 runner docker           376 Aug 13 06:59 pytest-cov.pth
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:59 pytest_cov
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 pytest_cov-2.10.0.dist-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:59 pytest_django
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 pytest_django-3.9.0.dist-info
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:59 pytest_html
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 pytest_html-2.1.1.dist-info
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:59 pytest_metadata
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 pytest_metadata-1.10.0.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 python_dateutil-2.8.1.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 python_whois-0.7.3-py3.7.egg-info
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:58 pytz
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 pytz-2020.1.dist-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:59 pywatchman
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 pywatchman-1.4.1-py3.7.egg-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:59 requests
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 requests-2.24.0.dist-info
drwxrwxrwx+ 11 runner docker          4096 Aug 13 06:58 rest_framework
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:59 sentry_sdk
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 sentry_sdk-0.16.2.dist-info
drwxrwxrwx+  6 runner runneradmin     4096 Aug  6 20:24 setuptools
drwxrwxrwx+  2 runner runneradmin     4096 Aug  6 20:24 setuptools-47.1.0.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 sh-1.12.0.dist-info
-rw-rw-rw-+  1 runner docker        109186 Aug 13 06:59 sh.py
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 six-1.15.0.dist-info
-rw-rw-rw-+  1 runner docker         34159 Aug 13 06:58 six.py
drwxrwxrwx+  5 runner docker          4096 Aug 13 06:58 sqlparse
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 sqlparse-0.3.1.dist-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 text_unidecode
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 text_unidecode-1.3.dist-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:59 toml
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 toml-0.10.1.dist-info
drwxrwxrwx+  6 runner docker          4096 Aug 13 06:58 traitlets
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 traitlets-4.3.3.dist-info
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:58 typed_ast
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 typed_ast-1.4.1.dist-info
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 typing_extensions-3.7.4.2.dist-info
-rw-rw-rw-+  1 runner docker         80218 Aug 13 06:58 typing_extensions.py
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 typing_inspect-0.6.0.dist-info
-rw-rw-rw-+  1 runner docker         24514 Aug 13 06:58 typing_inspect.py
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 tzlocal
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 tzlocal-2.1.dist-info
drwxrwxrwx+  6 runner docker          4096 Aug 13 06:59 urllib3
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 urllib3-1.25.10.dist-info
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:59 watchman
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:59 watchman-0.0.8.dist-info
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:58 wcwidth
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 wcwidth-0.2.5.dist-info
drwxrwxrwx+  4 runner docker          4096 Aug 13 06:59 whois
drwxrwxrwx+  3 runner docker          4096 Aug 13 06:58 yaml
drwxrwxrwx+  2 runner docker          4096 Aug 13 06:58 zipp-3.1.0.dist-info
-rw-rw-rw-+  1 runner docker          7121 Aug 13 06:58 zipp.py

sentry_sdk:

total 192
-rw-rw-rw-+ 1 runner docker   591 Aug 13 06:59 __init__.py
drwxrwxrwx+ 2 runner docker  4096 Aug 13 06:59 __pycache__
-rw-rw-rw-+ 1 runner docker  2419 Aug 13 06:59 _compat.py
-rw-rw-rw-+ 1 runner docker  2315 Aug 13 06:59 _functools.py
-rw-rw-rw-+ 1 runner docker  1128 Aug 13 06:59 _types.py
-rw-rw-rw-+ 1 runner docker  4704 Aug 13 06:59 api.py
-rw-rw-rw-+ 1 runner docker 13951 Aug 13 06:59 client.py
-rw-rw-rw-+ 1 runner docker  3487 Aug 13 06:59 consts.py
-rw-rw-rw-+ 1 runner docker  1132 Aug 13 06:59 debug.py
-rw-rw-rw-+ 1 runner docker  8213 Aug 13 06:59 envelope.py
-rw-rw-rw-+ 1 runner docker 21588 Aug 13 06:59 hub.py
drwxrwxrwx+ 5 runner docker  4096 Aug 13 06:59 integrations
-rw-rw-rw-+ 1 runner docker     0 Aug 13 06:59 py.typed
-rw-rw-rw-+ 1 runner docker 13893 Aug 13 06:59 scope.py
-rw-rw-rw-+ 1 runner docker 10507 Aug 13 06:59 serializer.py
-rw-rw-rw-+ 1 runner docker  7849 Aug 13 06:59 sessions.py
-rw-rw-rw-+ 1 runner docker 17976 Aug 13 06:59 tracing.py
-rw-rw-rw-+ 1 runner docker 11350 Aug 13 06:59 transport.py
-rw-rw-rw-+ 1 runner docker 24212 Aug 13 06:59 utils.py
-rw-rw-rw-+ 1 runner docker  4560 Aug 13 06:59 worker.py

The errors are basically around all site-packages. - It seems like it cannot read/find them.

belthaZornv commented 4 years ago

I'm out of solutions, let me know if you have anything from your end @gbleaney - thanks for your time dude.

gbleaney commented 4 years ago

Sorry, @belthaZornv I've been off work the last while and not checking anything from GitHub. Your config looks reasonable to me, so I'm not sure what's going wrong. I've asked @pradeep90 to take a look, since I'm out of ideas.

belthaZornv commented 4 years ago

Thank you @gbleaney!

pradeep90 commented 4 years ago

@shannonzhu Could you take a look at this? Is the search path not configured correctly?

shannonzhu commented 4 years ago

The configuration and search path field looks correct to me. Could you run pyre --noninteractive check and paste the full output here? I'm curious to see wha search path the pyre client is picking up and passing into the backend, in case that contains any additional clues.

Also, do you get a similar error if you try to import from a different module in this search path, like typed_ast? Just in case, could you share the path of the file containing this error and the import line triggering it?

belthaZornv commented 4 years ago

Good morning, to be entirely honest, I've ended up removing it - but you can reproduce with a simple repository on github, with a simple python app (with packages) and github actions that installs requirements.txt, and run pyre on it.

facebook / pyre-check

Github Actions and Import Errors #302