search

j-bennet / wharfee

A CLI with autocompletion and syntax highlighting for Docker commands.
BSD 3-Clause "New" or "Revised" License
657 stars 41 forks source link

failing behave tests #152

Closed Tset-Noitamotua closed 7 years ago

Tset-Noitamotua commented 7 years ago

I see some behave test failing on OS X El Capitan

(WHARFEE) ME@OSX[~/_GITHUB/wharfee]$ cd tests/
(WHARFEE) ME@OSX[~/_GITHUB/wharfee/tests]$ behave 

Scenario: list images                  # features/2_image_commands.feature:17
    Given we have wharfee installed      # features/steps/1_basic_commands.py:11 0.006s
    When we run wharfee                  # features/steps/1_basic_commands.py:20 0.003s
    And we wait for prompt               # features/steps/1_basic_commands.py:28 0.448s
    When we list images                  # features/steps/2_image_commands.py:92 0.104s
    Then we see image hello-world listed # features/steps/2_image_commands.py:100 30.005s
      Traceback (most recent call last):
        File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/model.py", line 1456, in run
          match.run(runner.context)
        File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/model.py", line 1903, in run
          self.func(context, *args, **kwargs)
        File "features/steps/2_image_commands.py", line 105, in step_see_image_listed
          wrappers.expect_exact(context, 'hello-world')
        File "/Users/ME/_GITHUB/wharfee/tests/features/steps/wrappers.py", line 30, in expect_exact
          handle_exception(context, expected)
        File "/Users/ME/_GITHUB/wharfee/tests/features/steps/wrappers.py", line 40, in handle_exception
          actual))
      Exception: Expected:
      ---
      hello-world
      ---

      Actual:
      ---
      imageswharfee> images
      Repository        Created       Tag                 VirtualSize    ParentId    Id
      ----------------  ------------  ------------------  -------------  ----------  -----------
      postgres          3 days ago    9.4                 252.675 MB                 sha256:4528
      redis             last week     alpine              20.057 MB                  sha256:9947
      nginx             3 weeks ago   latest              173.103 MB                 sha256:abf3
      python            4 weeks ago   2.7-alpine          68.9 MB                    sha256:9c8c
      microsoft/dotnet  last month    1.0.0-preview2-sdk  512.598 MB                 sha256:6704
      busybox           2 months ago  latest              1.043 MB                   sha256:e02e
      test-image        2 months ago  latest              1.043 MB                   sha256:e02e
      node              8 months ago  5.11.0-slim         197.795 MB                 sha256:cb88
      wharfee>
      ---

Scenario: inspect image             # features/2_image_commands.feature:24
    Given we have wharfee installed   # features/steps/1_basic_commands.py:11 0.008s
    When we run wharfee               # features/steps/1_basic_commands.py:20 0.002s
    And we wait for prompt            # features/steps/1_basic_commands.py:28 0.456s
    When we inspect image hello-world # features/steps/2_image_commands.py:108 0.105s
    Then we see "/hello" printed out  # features/steps/1_basic_commands.py:69 30.004s
      Traceback (most recent call last):
        File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/model.py", line 1456, in run
          match.run(runner.context)
        File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/model.py", line 1903, in run
          self.func(context, *args, **kwargs)
        File "features/steps/1_basic_commands.py", line 75, in step_see_output
          wrappers.expect_exact(context, patterns)
        File "/Users/ME/_GITHUB/wharfee/tests/features/steps/wrappers.py", line 30, in expect_exact
          handle_exception(context, expected)
        File "/Users/ME/_GITHUB/wharfee/tests/features/steps/wrappers.py", line 40, in handle_exception
          actual))
      Exception: Expected:
      ---
      [u'"/hello"', u'/hello']
      ---

      Actual:
      ---
      inspect hello-worldwharfee> inspect hello-world
      Container or image ID is required.
      wharfee>
      ---

Scenario: restart container                                                             # features/3_container_commands.feature:96
    Given we have wharfee installed                                                       # features/steps/1_basic_commands.py:11 0.007s
    When we run wharfee                                                                   # features/steps/1_basic_commands.py:20 0.003s
    And we wait for prompt                                                                # features/steps/1_basic_commands.py:28 0.493s
    When we run container foo with image busybox and command /bin/sh and options -d -i -t # features/steps/3_container_commands.py:8 0.109s
    And we wait for prompt                                                                # features/steps/1_basic_commands.py:28 0.003s
    Then we see "Interactive terminal is closed" printed out                              # features/steps/1_basic_commands.py:69 2.804s
    When we restart container foo                                                         # features/steps/3_container_commands.py:217 0.106s
    Then we see foo at line end                                                           # features/steps/1_basic_commands.py:78 2.041s
      Traceback (most recent call last):
        File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/model.py", line 1456, in run
          match.run(runner.context)
        File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/model.py", line 1903, in run
          self.func(context, *args, **kwargs)
        File "features/steps/1_basic_commands.py", line 83, in step_see_line_end
          wrappers.expect_exact(context, text + '
      ')
        File "/Users/ME/_GITHUB/wharfee/tests/features/steps/wrappers.py", line 30, in expect_exact
          handle_exception(context, expected)
        File "/Users/ME/_GITHUB/wharfee/tests/features/steps/wrappers.py", line 40, in handle_exception
          actual))
      Exception: Expected:
      ---
      foo

      ---

      Actual:
      ---
      .
      wharfee>
      restart foowharfee> restart foo
      UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=2)
      Goodbye!
      ---

Exception Exception: Expected:
---
wharfee> 
---

Actual:
---

---
Traceback (most recent call last):
  File "/Users/ME/.virtualenvs/WHARFEE/bin/behave", line 11, in <module>
    sys.exit(main())
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/__main__.py", line 109, in main
    failed = runner.run()
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/runner.py", line 672, in run
    return self.run_with_paths()
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/runner.py", line 693, in run_with_paths
    return self.run_model()
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/runner.py", line 483, in run_model
    failed = feature.run(self)
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/model.py", line 523, in run
    failed = scenario.run(runner)
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/model.py", line 919, in run
    runner.run_hook('after_scenario', runner.context, self)
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/runner.py", line 405, in run_hook
    self.hooks[name](context, *args)
  File "features/environment.py", line 40, in after_scenario
    wrappers.expect_exact(context, 'wharfee> ')
  File "/Users/ME/_GITHUB/wharfee/tests/features/steps/wrappers.py", line 30, in expect_exact
    handle_exception(context, expected)
  File "/Users/ME/_GITHUB/wharfee/tests/features/steps/wrappers.py", line 40, in handle_exception
    actual))
Exception: Expected:
---
wharfee> 
---

Actual:
---

---

Unit tests PASSED, just 1 SKIPPED

(WHARFEE) ME@OSX[~/_GITHUB/wharfee]$ py.test
============================================================================= test session starts ==============================================================================
platform darwin -- Python 2.7.12, pytest-3.0.5, py-1.4.32, pluggy-0.4.0
rootdir: /Users/ME/_GITHUB/wharfee, inifile: 
collected 79 items 

tests/test_client.py .
tests/test_completer.py ..........................................
tests/test_formatter.py ...s.....
tests/test_helpers.py .........
tests/test_options.py ..................

===================================================================== 78 passed, 1 skipped in 0.39 seconds =====================================================================

Cheers Tset

j-bennet commented 7 years ago

Ah yes. It looks like other tests are being run before hello-world image is downloaded. Have to make sure it's downloaded first.

j-bennet commented 7 years ago

These two commits fix the problem:

https://github.com/j-bennet/wharfee/commit/e774ce839f4c8bbf0daa76fb8838e8e2870fcb23 https://github.com/j-bennet/wharfee/commit/1fdae3873a283f8b04475cbebfd018e4970fee4d

Tset-Noitamotua commented 7 years ago

I have cloned the master branch again. Now I see one behave test fail:

  Scenario: restart container                                                             # features/3_container_commands.feature:96
    Given we have wharfee installed                                                       # features/steps/1_basic_commands.py:11 0.006s
    When we run wharfee                                                                   # features/steps/1_basic_commands.py:20 0.002s
    And we wait for prompt                                                                # features/steps/1_basic_commands.py:28 0.439s
    When we run container foo with image busybox and command /bin/sh and options -d -i -t # features/steps/3_container_commands.py:8 0.106s
    And we wait for prompt                                                                # features/steps/1_basic_commands.py:28 0.002s
    Then we see "Interactive terminal is closed" printed out                              # features/steps/1_basic_commands.py:69 2.757s
    When we restart container foo                                                         # features/steps/3_container_commands.py:225 0.106s
    Then we see foo restarted                                                             # features/steps/3_container_commands.py:201 2.092s
      Traceback (most recent call last):
        File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/model.py", line 1456, in run
          match.run(runner.context)
        File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/model.py", line 1903, in run
          self.func(context, *args, **kwargs)
        File "features/steps/3_container_commands.py", line 206, in step_see_restarted
          wrappers.expect(context, '{0}
      '.format(name), 60)
        File "/Users/ME/_GITHUB/wharfee/tests/features/steps/wrappers.py", line 17, in expect
          handle_exception(context, expected)
        File "/Users/ME/_GITHUB/wharfee/tests/features/steps/wrappers.py", line 40, in handle_exception
          actual))
      Exception: Expected:
      ---
      foo

      ---

      Actual:
      ---
      .
      wharfee>
      restart foowharfee> restart foo
      UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=2)
      Goodbye!
      ---

Exception Exception: Expected:
---
wharfee> 
---

Actual:
---

---
Traceback (most recent call last):
  File "/Users/ME/.virtualenvs/WHARFEE/bin/behave", line 11, in <module>
    sys.exit(main())
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/__main__.py", line 109, in main
    failed = runner.run()
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/runner.py", line 672, in run
    return self.run_with_paths()
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/runner.py", line 693, in run_with_paths
    return self.run_model()
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/runner.py", line 483, in run_model
    failed = feature.run(self)
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/model.py", line 523, in run
    failed = scenario.run(runner)
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/model.py", line 919, in run
    runner.run_hook('after_scenario', runner.context, self)
  File "/Users/ME/.virtualenvs/WHARFEE/lib/python2.7/site-packages/behave/runner.py", line 405, in run_hook
    self.hooks[name](context, *args)
  File "features/environment.py", line 40, in after_scenario
    wrappers.expect_exact(context, 'wharfee> ')
  File "/Users/ME/_GITHUB/wharfee/tests/features/steps/wrappers.py", line 30, in expect_exact
    handle_exception(context, expected)
  File "/Users/ME/_GITHUB/wharfee/tests/features/steps/wrappers.py", line 40, in handle_exception
    actual))
Exception: Expected:
---
wharfee> 
---

Actual:
---

---
j-bennet commented 7 years ago

Yikes. I noticed before that default timeout of 2s is not enough. Should probably up that. Thanks for the catch!

j-bennet commented 7 years ago

Those tests are now fixed in master. Container restart seems to be just too slow, so I disabled that.