nestorsalceda / mamba

The definitive testing tool for Python. Born under the banner of Behavior Driven Development (BDD).
http://nestorsalceda.github.io/mamba
MIT License
520 stars 64 forks source link

unittest fails under windows in travis and github workflows #152

Open Krasjet opened 4 years ago

Krasjet commented 4 years ago

Describe the bug

The unittest of mamba fails under windows in a continuous integration environment such as travis and github workflows. It seems to be related to test discoveries.

In github workflow, added in my testing fork:

 ........................F........F..............................................................

 2 examples failed of 96 ran in 0.5353 seconds

 Failures:

   1) ExampleCollector when loading with relative import it loads module and perform relative import
      Failure/Error: spec\example_collector_spec.py module = _load_module(WITH_RELATIVE_IMPORT_PATH)
          ImportError: attempted relative import with no known parent package

      File "spec\fixtures\with_relative_import.py", line 5, in <module>
          from .helpers import HelperClass
      File "d:\a\mamba\mamba\mamba\example_collector.py", line 70, in _module_from_ast
          exec(code, module.__dict__)
      File "d:\a\mamba\mamba\mamba\example_collector.py", line 52, in _load_module_from
          yield self._module_from_ast(name, path)
      File "C:\hostedtoolcache\windows\Python\3.8.5\x64\lib\contextlib.py", line 113, in __enter__
          return next(self.gen)
      File "d:\a\mamba\mamba\mamba\example_collector.py", line 20, in modules
          with self._load_module_from(path) as module:
      File "spec\example_collector_spec.py", line 36, in _load_module
          return list(example_collector.modules())[0]
      File "spec\example_collector_spec.py", line 167, in 00000043__it loads module and perform relative import
          module = _load_module(WITH_RELATIVE_IMPORT_PATH)

   2) Example it calculates elapsed time
      Failure/Error: spec\example_spec.py expect(self.example.elapsed_time.total_seconds()).to(be_above(0))
          AssertionError: expected: 0.0 to be above 0

      File "C:\Users\runneradmin\.virtualenvs\mamba-RRGFfIsI\lib\site-packages\expects\expectations.py", line 33, in _assert
          raise AssertionError(self._failure_message(matcher, reasons))
      File "C:\Users\runneradmin\.virtualenvs\mamba-RRGFfIsI\lib\site-packages\expects\expectations.py", line 25, in to
          self._assert(matcher)
      File "spec\example_spec.py", line 25, in 00000057__it calculates elapsed time
          expect(self.example.elapsed_time.total_seconds()).to(be_above(0))

In travis:


........................F..F.....F..............................................................

3 examples failed of 96 ran in 0.5156 seconds

Failures:

  1) ExampleCollector when loading with relative import it loads module and perform relative import
     Failure/Error: spec\example_collector_spec.py module = _load_module(WITH_RELATIVE_IMPORT_PATH)
         ImportError: attempted relative import with no known parent package

     File "spec\fixtures\with_relative_import.py", line 5, in <module>
         from .helpers import HelperClass
     File "c:\users\travis\build\krasjet\mamba\mamba\example_collector.py", line 70, in _module_from_ast
         exec(code, module.__dict__)
     File "c:\users\travis\build\krasjet\mamba\mamba\example_collector.py", line 52, in _load_module_from
         yield self._module_from_ast(name, path)
     File "c:\python38\lib\contextlib.py", line 113, in __enter__
         return next(self.gen)
     File "c:\users\travis\build\krasjet\mamba\mamba\example_collector.py", line 20, in modules
         with self._load_module_from(path) as module:
     File "spec\example_collector_spec.py", line 36, in _load_module
         return list(example_collector.modules())[0]
     File "spec\example_collector_spec.py", line 167, in 00000043__it loads module and perform relative import
         module = _load_module(WITH_RELATIVE_IMPORT_PATH)

  2) ExampleGroup when run it calculates elapsed time
     Failure/Error: spec\example_group_spec.py expect(self.example_group.elapsed_time.total_seconds()).to(be_above(0))
         AssertionError: expected: 0.0 to be above 0

     File "C:\Users\travis\.virtualenvs\mamba-GuhryZ-J\lib\site-packages\expects\expectations.py", line 33, in _assert
         raise AssertionError(self._failure_message(matcher, reasons))
     File "C:\Users\travis\.virtualenvs\mamba-GuhryZ-J\lib\site-packages\expects\expectations.py", line 25, in to
         self._assert(matcher)
     File "spec\example_group_spec.py", line 33, in 00000048__it calculates elapsed time
         expect(self.example_group.elapsed_time.total_seconds()).to(be_above(0))

  3) Example it calculates elapsed time
     Failure/Error: spec\example_spec.py expect(self.example.elapsed_time.total_seconds()).to(be_above(0))
         AssertionError: expected: 0.0 to be above 0

     File "C:\Users\travis\.virtualenvs\mamba-GuhryZ-J\lib\site-packages\expects\expectations.py", line 33, in _assert
         raise AssertionError(self._failure_message(matcher, reasons))
     File "C:\Users\travis\.virtualenvs\mamba-GuhryZ-J\lib\site-packages\expects\expectations.py", line 25, in to
         self._assert(matcher)
     File "spec\example_spec.py", line 25, in 00000057__it calculates elapsed time
         expect(self.example.elapsed_time.total_seconds()).to(be_above(0))

with the following config

language: python
jobs:
  include:
    - name: "windows"
      os: windows
      language: shell
      before_install:
        - choco install python --version 3.8.4
        - python -m pip install --upgrade pip
      env: PATH=/c/Python38:/c/Python38/Scripts:$PATH
install:
  - pip3 install pipenv
  - pipenv install --dev
script:
  - pipenv run mamba
after_success:
  - pipenv run coverage report

mamba is also not behaving correctly in the github workflow in one of my project.

admits_float

ToCFilter

extract_toc

FontFilter

BoundingBoxFilter

open_pdf:

ToCEntry

dump_toc

parse_toc

gen_toc

read_toc

write_toc

extract_meta:

dump_meta:

14 examples ran in 0.5021 seconds
##[error]Process completed with exit code 1.

There should be 66 examples in total, but only 14 are discovered.