jejacks0n / teaspoon

Teaspoon: Javascript test runner for Rails. Use Selenium, BrowserStack, or PhantomJS.
1.43k stars 244 forks source link

Teaspoon running very slow if running coverage #464

Closed apellizzn closed 8 years ago

apellizzn commented 8 years ago

Hi, have the following config in spec/teaspoon_env.rb

Teaspoon.configure do |config|
  config.mount_at = "/teaspoon"
  config.root = nil
  config.asset_paths = ["spec/javascripts"]
  config.fixture_paths = []

  config.suite do |suite|
    suite.use_framework :jasmine, "2.3.4"
    suite.matcher = "{spec/javascripts}/**/*_spec.{js}"
    suite.helper = "spec_helper"
    suite.boot_partial = "boot"
    suite.body_partial = "body"
  end

  config.use_coverage = false
  config.coverage do |coverage|
    coverage.reports = ['text']
    coverage.ignore = [%r{/template/}, %r{/gems/}, %r{/lib/ruby/gems/}, %r{/vendor/assets/}, %r{/support/}, %r{/plugins/}, %r{/(.+)_helper.}]
  end
end

If i run teaspoon from CLI the execution ends in ~1sec but if i try to add coverage, teaspoon coverage=default that its taking >30sec After debugging teaspoon code i found that code causing this delay is in teaspoon/driver/phantomjs.rb

def run(*args, &block)
        IO.popen([executable, *args].join(" ")) { |io|  io.each(&block) }

        unless $?.nil? || $?.success?
          raise Teaspoon::DependencyError.new("Failed to use phantomjs, which exited with status code: #{$?.exitstatus}")
       end
end

actually seems that each method called on io in IO.popen block (even io.count) is taking lot of time to return a value but only if i specify that i also want coverage Is it normal? Am i missing something?

jejacks0n commented 8 years ago

if you dig further into the code, the IO.popen call is only running a shell command -- in this case, it's phantomjs. The reason it takes longer to run with coverage is several fold, so I'll explain what's happening and you can dig into that further if you feel there's still something in Teaspoon that is running "very slow".

When you run Teaspoon locally, it'll use the dev environment by default, specifically to take advantage of the dev env asset caching -- this is to make load times better, but the load times are always about the time it takes to transpile/concat your asset files.

Now, when you run with coverage, there's no way to actually tell which lines are being executed in javascript, and so the javascript must be wrapped in some crazy looking code to enable call tracking, statement call counts etc. For this, istanbul is used, and if you're interested in what that looks like, or how long it might take to instrument one of your files, please use istanbul to instrument one of your javascript files and check the output.

Ok, so now you know what's happening.

Now, because istanbul is not being used for all of your assets in development, instrumented versions of your assets obviously won't be cached, and so when you run Teaspoon with coverage you're negating the potential dev env asset cache that you've built up locally.

If you're not excluding assets that you don't want instrumented, consider adding them to the ignore rules that are outlined in the coverage configuration documentation as well.

Hope that gives you some insight into the project and how you can dig in deeper.

apellizzn commented 8 years ago

Thank you for your good explanation, i will try to figure out how to speed this up based on what you said