search

gleb-sevruk / pycrunch-engine

NCrunch inspired tool for continuous testing Python
https://pycrunch.com
Other
57 stars 8 forks source link

do not inspect folders/files Excluded by pycharm #31

Open yarikoptic opened 3 years ago

yarikoptic commented 3 years ago

in my case 

[978310] 2020-10-27 09:06:30,089 - watchgod.watcher - WARNING - error walking file system: FileNotFoundError [Errno 2] No such file or directory: '/home/yoh/proj/datalad/datalad-master/venvs/dev3.5/lib/python3.5/os.py'
[978310] 2020-10-27 09:06:30,488 - watchgod.watcher - WARNING - error walking file system: FileNotFoundError [Errno 2] No such file or directory: '/home/yoh/proj/datalad/datalad-master/venvs/dev3.5/lib/python3.5/os.py'
[978310] 2020-10-27 09:06:30,897 - watchgod.watcher - WARNING - error walking file system: FileNotFoundError [Errno 2] No such file or directory: '/home/yoh/proj/datalad/datalad-master/venvs/dev3.5/lib/python3.5/os.py'
[978310] 2020-10-27 09:06:31,296 - watchgod.watcher - WARNING - error walking file system: FileNotFoundError [Errno 2] No such file or directory: '/home/yoh/proj/datalad/datalad-master/venvs/dev3.5/lib/python3.5/os.py'
[978310] 2020-10-27 09:06:31,694 - watchgod.watcher - WARNING - error walking file system: FileNotFoundError [Errno 2] No such file or directory: '/home/yoh/proj/datalad/datalad-master/venvs/dev3.5/lib/python3.5/os.py'
[978310] 2020-10-27 09:06:32,099 - watchgod.watcher - WARNING - error walking file system: FileNotFoundError [Errno 2] No such file or directory: '/home/yoh/proj/datalad/datalad-master/venvs/dev3.5/lib/python3.5/os.py'

keeps flooding the screen. venvs is completely excluded in pycharm

gleb-sevruk commented 3 years ago

Yeah, this currently needs to be in excluded in .pycrunch-config.yaml

discovery:
  exclusions:
  # do not search tests inside venv 
  - venvs
engine:
  runtime: pytest

I haven't researched if it is possible to load excluded folders via Intelij SDK

yarikoptic commented 3 years ago

THANKS! Is there inclusions, which is pretty much defaults to . and are considered before exclusions - i.e. they define what to consider and then exclusions exclude some "considered"? ;) Then for 99% of projects it would be just include: [package_name, tests] (tests just in case if they aren't embedded within package_name) and disregard all the exterior stuff which is not expected to be tested anyways.

(BTW: exclude and include IMHO would be a bit cleaner to use here)

yarikoptic commented 3 years ago

also, as example in the codebase shows:

  exclusions:
  - front/
  - front\
  - build/
  - test_discovery_specs_demo.py

there is difficulty of handling directories across platforms. So may be exclusions and paths should be first unified to be POSIX for uniform specification?

yarikoptic commented 3 years ago

Exclusions aren't enough. I have now

$> cat .pycrunch-config.yaml                                                                                
discovery:
  exclusions:
  - build
  - venvs
  - dist
  - doc
  - .
engine:
  cpu-cores: 4
  load-pytest-plugins: false
  multiprocessing-threshold: 4
  runtime: nose
  timeout: 0
env:
  DATALAD_TESTS_SSH: "1"

and I think it did not pick up tests from venvs, but then I still have lots of 

[1072016] 2020-10-27 12:44:34,068 - watchgod.watcher - WARNING - error walking file system: FileNotFoundError [Errno 2] No such file or directory: '/home/yoh/proj/datalad/datalad-master/venvs/dev3.5/lib/python3.5/os.py'
[1072016] 2020-10-27 12:44:34,467 - watchgod.watcher - WARNING - error walking file system: FileNotFoundError [Errno 2] No such file or directory: '/home/yoh/proj/datalad/datalad-master/venvs/dev3.5/lib/python3.5/os.py'
[1072016] 2020-10-27 12:44:34,869 - watchgod.watcher - WARNING - error walking file system: FileNotFoundError [Errno 2] No such file or directory: '/home/yoh/proj/datalad/datalad-master/venvs/dev3.5/lib/python3.5/os.py'

I think it is largely because there is no inclusions and the whole tree is still traversed while applying exclusions later on

        py_files = folder_path.glob('**/*.py')
        ...
        with ModuleCleanup() as cleanup:

            for py_file in py_files:
...

I guess "inclusions" could be easily implemented by chaining all the folder_path.glob(f'{inclusion}**/*.py') when multiple provided, thus not even going into places were it shouldn't go.

edit 3: unfortunately it is not sufficient. I guess watcher also manages to look elsewhere ... I did hardcode in above glob datalad/ and still was getting those WARNINGS from watcher

yarikoptic commented 3 years ago
For fun of it -- here is how my venvs look like ;): ```shell $> ls venvs build-v3.7.0a4-2497-g2f6fae6e510/ build-v3.8.0a4-388-g396e0a8d9dc/ dev-dbg/ build-v3.7.0a4-2498-ga8474d025ca/ build-v3.8.0b1/ dev-isolated/ build-v3.8.0/ build-v3.8.0b1-109-ga7072ff56e2/ dev-nosystem/ build-v3.8.0a1/ build-v3.8.0b1-159-g63429c839b1/ dev-sregistry/ build-v3.8.0a1-110-g06babb24225/ build-v3.8.0b1-171-g515a026eb06/ dev-trusty/ build-v3.8.0a1-14-g2848d9d2991/ build-v3.8.0b1-178-g3e133c401a5/ dev3/ build-v3.8.0a1-27-g181835d5a9b/ build-v3.8.0b1-179-gffcc161c753+cb083f/ dev3+containers/ build-v3.8.0a1-4-gca7d2933a38/ build-v3.8.0b1-180-gbf8cb318035/ dev3-install/ build-v3.8.0a1-55-g903567e4f54/ build-v3.8.0b1-180-gbf8cb318035+cb083f/ dev3-installfrompip/ build-v3.8.0a1-7-g85e102a2b09/ build-v3.8.0b1-181-g4137514b949/ dev3-isolated/ build-v3.8.0a2-97-g495da292255/ build-v3.8.0b1-183-gf22c4cf11d1/ dev3.5/ build-v3.8.0a3-76-gb4bcefe5fe6/ build-v3.8.0b1-184-gcb083f7cdf6/ dev3.7/ build-v3.8.0a4/ build-v3.8.0b2/ build-v3.8.0a4-297-g3880f263d29/ dev/ ```
gleb-britecore commented 3 years ago

It looks like some symlink (lib/python3.5/os.py).

But I do not understand why is it watching this folder. Maybe there is some coverage collected on the file?

Can you try to put this as exclusion in: https://github.com/gleb-sevruk/pycrunch-engine/blob/27bbacc348cd073f2bef239ea5f19980d311b062/pycrunch/child_runtime/exclusions.py#L9

gleb-britecore commented 3 years ago

Maybe your tests hit lib/python3.5/os.py and then scheduled to be watched for changes inside ./pycrunch/watcher/fs_watcher.py FSWatcher.watch method

yarikoptic commented 3 years ago

I bet before I removed that venvs/dev3 it could have been hitting system wide python3.5 os . But the point was - why to watch files outside of the project at all? Sure they could change with upgrade, but I wouldn't expect them to -- should at least to be an option I guess (or configurable via inclusions thus limiting only to local project tree)

gleb-britecore commented 3 years ago

This is sometimes convenient to dive into libraries in pycharm and see what is covered inside of libraries: /Users/gleb/venv/lib/python3.6/site-packages/django/test/client.py: image

But I agree this should not default behavior.

yarikoptic commented 3 years ago

Yeah, indeed very nice. We never dared to dive that deep in the coverage (and very rarely troubleshooted bugs in python itself), sand since upload only coverage for our package to codecov, never collected coverage outside.