search

jejacks0n / teaspoon

Teaspoon: Javascript test runner for Rails. Use Selenium, BrowserStack, or PhantomJS.
1.43k stars 243 forks source link

Can't load fixtures when running from command line #435

Closed jaredmoody closed 8 years ago

jaredmoody commented 8 years ago

I'm having trouble running teaspoon on the command line at all, while everything is great in the browser:

> teaspoon -v
1.1.1

> teaspoon
Starting the Teaspoon server...
Teaspoon running default suite at http://127.0.0.1:63423/teaspoon/default
FUnable to load fixture "books.html".
/Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/formatter/base.rb:148:in `filename': undefined method `split' for nil:NilClass (NoMethodError)
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/formatter/modules/report_module.rb:13:in `block in log_error'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/formatter/modules/report_module.rb:11:in `each'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/formatter/modules/report_module.rb:11:in `log_error'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/formatter/base.rb:54:in `error'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/runner.rb:29:in `block in notify_formatters'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/runner.rb:29:in `each'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/runner.rb:29:in `notify_formatters'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/runner.rb:16:in `process'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/driver/phantomjs.rb:28:in `block in run_specs'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/driver/phantomjs.rb:35:in `each'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/driver/phantomjs.rb:35:in `block in run'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/driver/phantomjs.rb:35:in `popen'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/driver/phantomjs.rb:35:in `run'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/driver/phantomjs.rb:27:in `run_specs'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:55:in `run_specs'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:44:in `block in execute_without_handling'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:42:in `each'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:42:in `inject'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:42:in `execute_without_handling'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:27:in `execute'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:23:in `failures?'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/lib/teaspoon/command_line.rb:15:in `initialize'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/bin/teaspoon:6:in `new'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/teaspoon-1.1.1/bin/teaspoon:6:in `<top (required)>'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:268:in `load'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:268:in `block in load'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:240:in `load_dependency'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:268:in `load'
    from /Users/jaredmoody/Sites/albumexposure/albumexposure.com/bin/teaspoon:7:in `<top (required)>'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:268:in `load'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:268:in `block in load'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:240:in `load_dependency'
    from /Users/jaredmoody/.gem/ruby/2.2.3/gems/activesupport-4.2.5/lib/active_support/dependencies.rb:268:in `load'
    from /Users/jaredmoody/.rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /Users/jaredmoody/.rubies/ruby-2.2.3/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from -e:1:in `<main>'

books.html exists and is at spec/javascripts/fixtures, and that fixture path is specified in my teaspoon_env.rb:

Here's my full teaspoon_env.rb

Teaspoon.configure do |config|
  config.mount_at = "/teaspoon"
  config.root = nil
  config.asset_paths = ["spec/javascripts", "spec/javascripts/stylesheets"]
  config.fixture_paths = ["spec/javascripts/fixtures"]

  config.suite do |suite|
    suite.use_framework :mocha, "2.3.3"
    suite.matcher = "{spec/javascripts,app/assets}/**/*_spec.{js,js.coffee,coffee}"
    suite.helper = "spec_helper"
    suite.boot_partial = "boot"
    suite.body_partial = "body"
  end
end

I don't know if it's related, but I can't run them via rake either:

> rake teaspoon                                                                                                 ruby-2.2.3
Starting the Teaspoon server...
Teaspoon running default suite at http://127.0.0.1:63519/teaspoon/default
ReferenceError: Can't find variable: Teaspoon
  # /teaspoon/default?reporter=Console:13 -- global code

ReferenceError: Can't find variable: Teaspoon
  # /teaspoon/default?reporter=Console:99 -- global code

Failed to load: http://127.0.0.1:63519/teaspoon/default?reporter=Console
rake teaspoon failed

RAILS_ENV=test rake teaspoon also yields the Unable to load fixture failure.

mikepack commented 8 years ago

@jaredmoody thanks for reporting this bug. I'll try to spend some turkey time looking into it.

elad-maimon commented 8 years ago

I have the same issue for quite a time actually. One thing I noticed is that it happens only when I'm using phantomjs. If I switch to selenium it works.

Another wired symptom, which I'm not sure if it's related or not but may help, is that although I have only 25 tests in the file, when running from CLI it shows 68 tests (some are repeated). On the web UI or in the CLI with selenium it shows 25.

mikepack commented 8 years ago

@jaredmoody can you confirm this isn't related to any of the issues here? https://github.com/modeset/teaspoon/search?q=Can%27t+find+variable%3A+Teaspoon&type=Issues&utf8=%E2%9C%93

jaredmoody commented 8 years ago

@mikepack I don't think so. The only cases that seemed like they might be the same were the issues with assets, but I don't have an asset host in the test environment and both RAILS_ENV=test teaspoon and RAILS_ENV=test rake teaspoon fail with the same "undefined method Split" error.

I may be having one of those issues as well in the development environment, but looks like a different issue in test environment to me.

estebanz01 commented 8 years ago

+1 Here. I found that if I use phantomjs version 2.0, I got the same error reported here:

Starting the Teaspoon server...
Teaspoon running default suite at http://127.0.0.1:55531/teaspoon/default
swfobject failed to load to vhlarccontainer_content
Unable to load fixture "activity_item_player.html".
  # teaspoon-jasmine1.js:268
/home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/formatter/base.rb:148:in `filename': undefined method `split' for nil:NilClass (NoMethodError)
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/formatter/modules/report_module.rb:13:in `block in log_error'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/formatter/modules/report_module.rb:11:in `each'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/formatter/modules/report_module.rb:11:in `log_error'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/formatter/base.rb:54:in `error'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/runner.rb:29:in `block in notify_formatters'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/runner.rb:29:in `each'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/runner.rb:29:in `notify_formatters'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/runner.rb:16:in `process'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/driver/phantomjs.rb:28:in `block in run_specs'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/driver/phantomjs.rb:35:in `each'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/driver/phantomjs.rb:35:in `block in run'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/driver/phantomjs.rb:35:in `popen'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/driver/phantomjs.rb:35:in `run'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/driver/phantomjs.rb:27:in `run_specs'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:55:in `run_specs'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:44:in `block in execute_without_handling'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:42:in `each'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:42:in `inject'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:42:in `execute_without_handling'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:27:in `execute'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/console.rb:23:in `failures?'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/lib/teaspoon/command_line.rb:15:in `initialize'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/bin/teaspoon:6:in `new'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/gems/teaspoon-1.1.1/bin/teaspoon:6:in `<top (required)>'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/bin/teaspoon:19:in `load'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/bin/teaspoon:19:in `<main>'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/bin/ruby_executable_hooks:15:in `eval'
        from /home/ezapata/.rvm/gems/ruby-1.9.3-p545-railsexpress@mae/bin/ruby_executable_hooks:15:in `<main>'

but if I change phantomjs to 1.9.8, spec runs as expected:

Starting the Teaspoon server...
Teaspoon running default suite at http://127.0.0.1:41005/teaspoon/default
SyntaxError: Parse error

swfobject failed to load to vhlarccontainer_content
.....

Finished in 0.13200 seconds
5 examples, 0 failures

EDIT: Here's the uname -a command output in my Ubuntu 14.04 machine:

Linux ezapata-vmachine 3.13.0-74-generic #118-Ubuntu SMP Thu Dec 17 22:52:10 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
coaxial commented 8 years ago

I ran into this issue and here is what I found:

It seems the issue lies somewhere with PhantomJS. This is a binary of phantomjs 2.0.0 from a machine where rake teaspoon worked: http://s000.tinyupload.com/index.php?file_id=42632369713235520993 (sorry for the host, feel free to check the file for viruses before running it).

This binary is not the latest phantomjs 2.0. If I install the latest with brew install phantomjs, it says it installs 2.0.1 although phantomjs -v still outputs 2.0.0. Anyway, with the latest 2.0, the fixtures can't be loaded just like in this issue. But if you drop the binary linked above in /usr/local/Cellar/phantomjs/2.0.1/bin/ and run rake teaspoon again, everything works fine.

estebanz01 commented 8 years ago

When you put the binay in .../phantomjs/2.0.1/bin are you really sure that teaspoon is using that phantom version @Coaxial ?

coaxial commented 8 years ago

There is no other phantomjs binary installed on my machine so if it uses phantom at all, it would have to be the binary in that directory

estebanz01 commented 8 years ago

you should check. Maybe is loading the phantomjs somewhere else

coaxial commented 8 years ago

I did check, I searched the whole filesystem for a phantomjs binary and the only one coming up is the one installed by brew that I replaced with the version I linked above.

Besides, using the installed phantomjs in its 2.0.1 version from the brew command triggers the problem discussed in this issue. Replacing the binary with the one linked above lets teaspoon run without issues.

jejacks0n commented 8 years ago

So it sounds like a problem when phantomjs 2.0.1? That's helpful. I'll update and see if it's reproducible, and see if it's resolvable. Thanks. For now, I'd say use 2.0.0 if that's working correctly.

Jeremy Jackson

On Jan 13, 2016, at 12:36 PM, Pierre notifications@github.com wrote:

I did check, I searched the whole filesystem for a phantomjs binary and the only one coming up is the one installed by brew that I replaced with the version I linked above.

Besodes, using the installed phantomjs in its 2.0.1 version from the brew command triggers the problem discussed in this issue. Replacing the binary with the one linked above lets teaspoon run without issues.

— Reply to this email directly or view it on GitHub.

coaxial commented 8 years ago

I can't get 2.0.0 to compile on OS X 10.11, and the 2.0.0 from the official website doesn't uncompress with upx following their instructions. I give up, I'll just drop in the binary from tinyupload for now.

jejacks0n commented 8 years ago

well, it's clearly not a teaspoon issue directly, but I'll still try to figure it out.

if you have npm installed, consider sudo npm -g install phantomjs2 as an alternative.. looks like brew is failing, but npm just installs the binary precompiled.

estebanz01 commented 8 years ago

I built the phantomjs from source in Ubuntu and tested. Indeed is the 2.0.0 version with teaspoon in 1.1.1. I'll try later with 1.1.2 to see what would happen.

jejacks0n commented 8 years ago

2.0.0, or 2.0.1?

estebanz01 commented 8 years ago

the latest tag version of phantom is 2.0.0 https://github.com/ariya/phantomjs/tags https://github.com/ariya/phantomjs/blob/master/src/consts.h#L38

estebanz01 commented 8 years ago

I got the following output when updated to teaspoon 1.1.2

Unable to load fixture "activity_item_player.html".
  # teaspoon-jasmine1.js:268
  # :0 -- send
  # teaspoon-jasmine1.js:357 -- xhrRequest
  # teaspoon-jasmine1.js:263 -- load
  # teaspoon-jasmine1.js:217 -- load
  # teaspoon-jasmine1.js:1215 -- load
  # relative_path/to/spec_with_fixture.js:14
  # jasmine/1.3.1.js:1066 -- execute
  # jasmine/1.3.1.js:2098 -- next_
  # jasmine/1.3.1.js:2051 -- start
  # jasmine/1.3.1.js:2378 -- execute
  # jasmine/1.3.1.js:2098 -- next_
  # jasmine/1.3.1.js:2051 -- start
  # jasmine/1.3.1.js:2523 -- execute
  # jasmine/1.3.1.js:2098 -- next_
  # jasmine/1.3.1.js:2094 -- onComplete
  # jasmine/1.3.1.js:2480 -- finish
  # jasmine/1.3.1.js:2524
  # jasmine/1.3.1.js:2108 -- next_
  # jasmine/1.3.1.js:2094 -- onComplete
  # jasmine/1.3.1.js:2480 -- finish
  # jasmine/1.3.1.js:2524
  # jasmine/1.3.1.js:2108 -- next_
  # jasmine/1.3.1.js:2094 -- onComplete
  # jasmine/1.3.1.js:2480 -- finish
  # jasmine/1.3.1.js:2524
  # jasmine/1.3.1.js:2108 -- next_
  # jasmine/1.3.1.js:2088
jejacks0n commented 8 years ago

ah, I wonder if something changed about the XHR implementation or something.. ugg. browsers.

coaxial commented 8 years ago

I think the 2.0.1 version is a fix by @Vitallium for compiling issues with brew: https://github.com/Vitallium/phantomjs/releases/tag/2.0.1 so users still have a phantomjs option when running El Capitan and waiting for 2.1.

estebanz01 commented 8 years ago

I just compiled from source https://github.com/Vitallium/phantomjs/releases/tag/2.0.1 and this error, for me in Ubuntu, still happening.

jaredmoody commented 8 years ago

FWIW I'm getting this error with phantomjs 2.0.0

Sorry, it's confusing because 2.0.1 from homebrew prints out 2.0.0 for phantomjs -v

Confirmed that 2.0.0 works for me and 2.0.1 gives this error.

coaxial commented 8 years ago

phantomjs -v isn't reliable, it says 2.0.0 for both 2.0.0 and 2.0.1

vitallium commented 8 years ago

Hi guys. Someone mentioned me in a comment. PhantomJS dev here.

I found the problem yesterday. And can confirm that the problem is in AJAX requests. New feature for capturing the content of requests sometimes breaks AJAX requests. It removes the body and status codes.

I'll fix it ASAP.

Thanks!

jejacks0n commented 8 years ago

That's great, thanks so much for checking in and resolving it!

Jeremy Jackson

On Jan 14, 2016, at 11:13 PM, Vitaly Slobodin notifications@github.com wrote:

Hi guys. Someone mentioned me in a comment. PhantomJS dev here.

I found the problem yesterday. And can confirm that the problem is in AJAX requests. New feature for capturing the content of requests sometimes breaks AJAX requests. It removes the body and status codes.

I'll fix it ASAP.

Thanks!

— Reply to this email directly or view it on GitHub.

estebanz01 commented 8 years ago

Thanks! That's awesome! :+1:

coaxial commented 8 years ago

Thanks!

jejacks0n commented 8 years ago

I'm closing this, as it will probably be resolved shortly and will only effect a small amount of people who have installed the version where ajax isn't behaving. Thanks everybody for the info, and thanks @Vitallium for checking in before I had to really dig into it. =)