search

mislav / hub

A command-line tool that makes git easier to use with GitHub.
https://hub.github.com/
MIT License
22.79k stars 2.2k forks source link

test-all doesn't pass #2506

Open giuscri opened 4 years ago

giuscri commented 4 years ago

Command attempted: $ make test-all

What happened: After cloning the repository I launched make test-all expecting all tests to pass.

$ make test-all
script/bootstrap
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin19]
Using rake 12.3.3
Following files may not be writable, so sudo is needed:
  /Library/Ruby/Gems/2.6.0
  /Library/Ruby/Gems/2.6.0/build_info
  /Library/Ruby/Gems/2.6.0/cache
  /Library/Ruby/Gems/2.6.0/doc
  /Library/Ruby/Gems/2.6.0/extensions
  /Library/Ruby/Gems/2.6.0/gems
  /Library/Ruby/Gems/2.6.0/specifications
Using childprocess 1.0.1
Using contracts 0.16.0
Using builder 3.2.3
Using backports 3.15.0
Using cucumber-tag_expressions 1.1.1
Using gherkin 5.1.0
Using cucumber-core 3.2.1
Using cucumber-expressions 6.0.1
Using cucumber-wire 0.0.1
Using diff-lcs 1.3
Using multi_json 1.13.1
Using multi_test 0.1.2
Using cucumber 3.1.2
Using ffi 1.11.1
Using rspec-support 3.8.2
Using rspec-expectations 3.8.4
Using thor 0.20.3
Using aruba 1.0.0.pre.alpha.4
Using bundler 1.17.2
Using mustermann 1.0.3
Using rack 2.0.7
Using rack-protection 2.0.5
Using tilt 2.0.9
Using sinatra 2.0.5
Bundle complete! 3 Gemfile dependencies, 25 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
Everything OK.
script/test
?       github.com/github/hub   [no test files]
ok      github.com/github/hub/cmd       (cached)
ok      github.com/github/hub/commands  (cached)
?       github.com/github/hub/coverage  [no test files]
?       github.com/github/hub/fixtures  [no test files]
ok      github.com/github/hub/git       (cached)
ok      github.com/github/hub/github    (cached)
?       github.com/github/hub/md2roff   [no test files]
?       github.com/github/hub/md2roff-bin       [no test files]
ok      github.com/github/hub/ui        (cached)
ok      github.com/github/hub/utils     (cached)
?       github.com/github/hub/version   [no test files]
tmux 3.0
Using the all profile...
..................................................................................................................................................................................................................................................................................................................F---......................F-......................................................................................................................................................F--............................................F..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................P----P--P---P--.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................F............F

(::) pending steps (::)

features/support/completion.rb:232:in `/^my shell is (\w+)$/'

features/support/completion.rb:232:in `/^my shell is (\w+)$/'

features/support/completion.rb:232:in `/^my shell is (\w+)$/'

features/support/completion.rb:232:in `/^my shell is (\w+)$/'

(::) failed steps (::)

expected that command "hub create" has exit status of "0", but has "1". (RSpec::Expectations::ExpectationNotMetError)
features/authentication.feature:35:in `And the exit status should be 0'

expected that command "hub create" has exit status of "0", but has "1". (RSpec::Expectations::ExpectationNotMetError)
features/authentication.feature:122:in `And the exit status should be 0'

expected [#<Command "git init --quiet  'dotfiles'" exited:0>, #<Command "hub create" exited:1>] to include an object have output: output string is eq: "open /InvalidConfigFile: permission denied\\n"

   object at index 0 failed to match:
      expected `git init --quiet  'dotfiles'` to have output output string is eq: "open /InvalidConfigFile: permission denied\\n"
      but was:

   object at index 1 failed to match:
      expected `hub create` to have output output string is eq: "open /InvalidConfigFile: permission denied\\n"
      but was:
         open /InvalidConfigFile: read-only file system (RSpec::Expectations::ExpectationNotMetError)
features/authentication.feature:507:in `Then the output should contain exactly:'

timeout while waiting for completions to expand (RuntimeError)
/Users/g/hub/features/support/completion.rb:186:in `tmux_wait_for_completion'
/Users/g/hub/features/support/completion.rb:221:in `tmux_completion_menu_basic'
/Users/g/hub/features/support/completion.rb:274:in `/^the completion menu should offer "([^"]+?)"( unsorted)?$/'
features/bash_completion.feature:39:in `Then the completion menu should offer "--no-remote --remote-name --org" unsorted'

expected: {"-F"=>"file", "-M"=>"milestone", "-a"=>"user", "-b"=>"base", "-f"=>"force (skip check for local commits)", "-h"=>"head", "-i"=>"issue", "-l"=>"labels", "-m"=>"message"}
     got: {"--no-edit"=>"use first commit message for pull request title/description", "-F"=>"file", "-M"=>"mil...force (skip check for local commits)", "-h"=>"head", "-i"=>"issue", "-l"=>"labels", "-m"=>"message"}

(compared using ==)

Diff:
@@ -1,3 +1,4 @@
+"--no-edit" => "use first commit message for pull request title/description",
 "-F" => "file",
 "-M" => "milestone",
 "-a" => "user",
 (RSpec::Expectations::ExpectationNotMetError)
/Users/g/hub/features/support/completion.rb:290:in `/^the completion menu should offer:/'
features/zsh_completion.feature:22:in `Then the completion menu should offer:'

timeout while waiting for completions to expand (RuntimeError)
/Users/g/hub/features/support/completion.rb:186:in `tmux_wait_for_completion'
/Users/g/hub/features/support/completion.rb:294:in `/^the command should expand to "(.+?)"$/'
features/zsh_completion.feature:45:in `Then the command should expand to "git ci-status"'

Failing Scenarios:
cucumber -p all features/authentication.feature:5 # Scenario: Ask for username & password, create authorization
cucumber -p all features/authentication.feature:88 # Scenario: Rename & retry creating authorization if there's a token name collision
cucumber -p all features/authentication.feature:504 # Scenario: Config file is not writeable, should exit before asking for credentials
cucumber -p all features/bash_completion.feature:36 # Scenario: Completion of fork argument
cucumber -p all features/zsh_completion.feature:20 # Scenario: Completion of pull-request arguments
cucumber -p all features/zsh_completion.feature:42 # Scenario: "ci-" expands to "ci-status"

544 scenarios (6 failed, 4 pending, 534 passed)
3400 steps (6 failed, 17 skipped, 4 pending, 3373 passed)
1m50.244s
make: *** [test-all] Error 1

More info: HEAD is 2cff4c2adc24a5fa150a471d44f2cfd3ad019c3c

mislav commented 4 years ago

@giuscri Thank you for reporting!

Automated shell completion tests were historically very brittle and I'm considering disabling them by default when running the test suite.

The failures in features/authentication.feature are interesting: one seems to be a mismatch in error message expectation, but I can't guess the cause of others.