search

lsegal / yard

YARD is a Ruby Documentation tool. The Y stands for "Yay!"
http://yardoc.org
MIT License
1.94k stars 397 forks source link

test failures on Debian with Ruby 1.8 and 1.9.3 #397

Closed lnussbaum closed 12 years ago

lnussbaum commented 12 years ago

Hi,

While creating a Debian package for yard, I ran into the following spec failures:

Ruby 1.8

Failures:

1) YARD::Templates::Helpers::HtmlHelper#htmlify should autolink URLs (markdown specific) Failure/Error: Unable to find matching line from backtrace expected: "

<a href=\"http://example.com\">http://example.com

" got: "

http://example.com

" (using ==)

./spec/templates/helpers/html_helper_spec.rb:133

 # debian/ruby-tests.rb:2

2) YARD::CLI::Yardoc#find_object should use cache if available Failure/Error: Unable to find matching line from backtrace <File (class)> received :exist? with unexpected arguments expected: (".yardoc") got: ("foo")

./lib/yard/cli/yri.rb:115:in `find_object'

 # ./lib/yard/cli/yri.rb:114:in`each'
 # ./lib/yard/cli/yri.rb:114:in `find_object'
 # ./spec/cli/yri_spec.rb:21
 # debian/ruby-tests.rb:2

3) YARD::CLI::Yardoc#find_object should never use cache ahead of current directory's .yardoc Failure/Error: Unable to find matching line from backtrace <File (class)> received :exist? with unexpected arguments expected: (".yardoc") got: ("foo")

./lib/yard/cli/yri.rb:115:in `find_object'

 # ./lib/yard/cli/yri.rb:114:in`each'
 # ./lib/yard/cli/yri.rb:114:in `find_object'
 # ./spec/cli/yri_spec.rb:30
 # debian/ruby-tests.rb:2

4) YARD::CLI::Yardoc#run should search for objects and print their documentation Failure/Error: Unable to find matching line from backtrace SystemExit: exit

./lib/yard/cli/yri.rb:65:in `exit'

 # ./lib/yard/cli/yri.rb:65:in`run'
 # ./spec/cli/yri_spec.rb:71
 # debian/ruby-tests.rb:2

Finished in 10.41 seconds 1265 examples, 4 failures, 1 pending

Ruby 1.9.3 (rc1)

1) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid class 'CallMethod('test')' Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:65:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:65:in`block (3 levels) in <top (required)>'

2) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid class 'VSD^#}}' Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:65:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:65:in`block (3 levels) in <top (required)>'

3) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid class 'not.aclass' Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:65:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:65:in`block (3 levels) in <top (required)>'

4) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid class 'self' Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:65:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:65:in`block (3 levels) in <top (required)>'

5) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid superclass '@@INVALID' but it should create the class. Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:72:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:72:in`block (3 levels) in <top (required)>'

6) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid superclass 'hi' but it should create the class. Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:72:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:72:in`block (3 levels) in <top (required)>'

7) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid superclass '$MYCLASS' but it should create the class. Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:72:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:72:in`block (3 levels) in <top (required)>'

8) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid superclass 'AnotherClass.new' but it should create the class. Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:72:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:72:in`block (3 levels) in <top (required)>'

9) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError if the constant class reference 'class << SomeConstant' does not point to a valid class name Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:83:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:79:in`block (3 levels) in <top (required)>'

10) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError if the constant class reference 'class << SomeConstant' does not point to a valid class name Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:83:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:79:in`block (3 levels) in <top (required)>'

11) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError if the constant class reference 'class << SomeConstant' does not point to a valid class name Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:83:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:79:in`block (3 levels) in <top (required)>'

12) YARD::Handlers::Ruby::MethodHandler should raise an undocumentable error when a method is defined on an object instance Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/method_handler_spec.rb:127:in`block (2 levels) in <top (required)>'

13) YARD::Templates::Helpers::HtmlHelper#htmlify should autolink URLs (markdown specific) Failure/Error: Unable to find matching line from backtrace expected: "

<a href=\"http://example.com\">http://example.com

" got: "

http://example.com

" (using ==)

./spec/templates/helpers/html_helper_spec.rb:133:in `block (3 levels) in <top (required)>'

14) YARD::CLI::Yardoc#find_object should use cache if available Failure/Error: Unable to find matching line from backtrace <File (class)> received :exist? with unexpected arguments expected: (".yardoc") got: ("foo")

./lib/yard/cli/yri.rb:115:in `block in find_object'

 # ./lib/yard/cli/yri.rb:114:in`each'
 # ./lib/yard/cli/yri.rb:114:in `find_object'
 # ./spec/cli/yri_spec.rb:21:in`block (3 levels) in <top (required)>'

15) YARD::CLI::Yardoc#find_object should never use cache ahead of current directory's .yardoc Failure/Error: Unable to find matching line from backtrace <File (class)> received :exist? with unexpected arguments expected: (".yardoc") got: ("foo")

./lib/yard/cli/yri.rb:115:in `block in find_object'

 # ./lib/yard/cli/yri.rb:114:in`each'
 # ./lib/yard/cli/yri.rb:114:in `find_object'
 # ./spec/cli/yri_spec.rb:30:in`block (3 levels) in <top (required)>'

16) YARD::CLI::Yardoc#run should search for objects and print their documentation Failure/Error: Unable to find matching line from backtrace SystemExit: exit

./lib/yard/cli/yri.rb:65:in `exit'

 # ./lib/yard/cli/yri.rb:65:in`run'
 # ./spec/cli/yri_spec.rb:71:in `block (3 levels) in <top (required)>'

Finished in 6.98 seconds 1313 examples, 16 failures, 1 pending

Thanks

Lucas

lsegal commented 12 years ago

Hi Lucas,

(FYI my tests are with Ubuntu, but they should be similar)

Most of these failures look related to file load order. If you run certain spec files before others, things fail. They shouldn't, but occasionally there's some environment modification going on. You should be able to run these files individually and have them pass, in the worst case. Specifically, if you run rspec spec/handlers/class_handler_spec.rb, it should pass. It seems that rspec chooses different file orders in Debian for some extremely odd reason, because neither OSX nor Windows (imagine that!) run into this problem. Either way, this is just a test env issue, not a problem with the actual code.

As for the failures in yri_spec.rb, I am not reproducing those-- this might be due to the specs failing and leaving garbage around in the repo. Does git status show anything funky? If so, reset --hard and try that spec file individually.

And finally I am also not reproducing your html_helper_spec.rb failure. That specific test depends on RDiscount being used, or any markdown lib that performs autolinking. You may have an older version of rdiscount and/or you didn't bundle install prior to testing, which would install the lib for you. If you install/update rdiscount to 1.6.8 and run that spec, does it still fail?

lsegal commented 12 years ago

Note that I'm not closing this ticket because we should work on making the specs more resilient to load order issues-- I'm going to give it a shot. In the worst case, we'll have to forcibly lock down the load order in the rake task.

graaff commented 12 years ago

FWIW the html_helper_spec.rb failure occurs for us on Gentoo with jruby since rdiscount isn't available for it. We now ignore that spc for jruby in our ebuild.

lsegal commented 12 years ago

@graaff doesn't jruby support loading of C exts now? those tests should be marked as pending if rdiscount isn't up, anyway... does the pending test throw off your CI? or is it failing?

lsegal commented 12 years ago

@ln Note that I've only targeted fixes in 1.9.3. I still could not reproduce all the failures. If there are remaining test breakages in Debian (or any distro) for 1.9.3 or 1.8, can you open another ticket for those specific failures? Thanks!

lnussbaum commented 12 years ago

Hi,

Using the "master" git tree, I still get the following failures:

=== Ruby1.8

Pending: YARD::Server::RackMiddleware should handle requests

rack required for these tests

#

YARD::Server::RackMiddleware should pass up to the next middleware on 404

rack required for these tests

#

YARD::Templates::Helpers::HtmlHelper#htmlify should not use hard breaks for textile markup (RedCloth specific)

test requires redcloth gem

#

YARD::Templates::Helpers::HtmlHelper#htmlify should use hard breaks for textile_strict markup (RedCloth specific)

test requires redcloth gem

#

YARD::CLI::Server should accept -a rack to create Rack adapter

rack required for this test

#

YARD::CLI::Server should default to Rack adapter if exists on system

rack required for this test

#

Failures:

1) YARD::Registry.yardoc_file_for_gem should return local .yardoc path for gem if for_writing=true and dir is not writable Failure/Error: Unable to find matching line from backtrace Errno::EACCES: Permission denied - /sbuild-nonexistent

./lib/yard/registry.rb:378:in `local_yardoc_file'

 # ./lib/yard/registry.rb:62:in`yardoc_file_for_gem'
 # ./spec/registry_spec.rb:56
 # debian/ruby-tests.rb:2

2) YARD::CLI::Yardoc#run should search for objects and print their documentation Failure/Error: Unable to find matching line from backtrace SystemExit: exit

./lib/yard/cli/yri.rb:65:in `exit'

 # ./lib/yard/cli/yri.rb:65:in`run'
 # ./spec/cli/yri_spec.rb:71
 # debian/ruby-tests.rb:2

Finished in 8.63 seconds 1327 examples, 2 failures, 6 pending

=== Ruby1.9.3

Pending: YARD::Server::RackMiddleware should handle requests

rack required for these tests

#

YARD::Server::RackMiddleware should pass up to the next middleware on 404

rack required for these tests

#

YARD::Templates::Helpers::HtmlHelper#htmlify should not use hard breaks for textile markup (RedCloth specific)

test requires redcloth gem

#

YARD::Templates::Helpers::HtmlHelper#htmlify should use hard breaks for textile_strict markup (RedCloth specific)

test requires redcloth gem

#

YARD::CLI::Server should accept -a rack to create Rack adapter

rack required for this test

#

YARD::CLI::Server should default to Rack adapter if exists on system

rack required for this test

#

Failures:

1) YARD::Registry.yardoc_file_for_gem should return local .yardoc path for gem if for_writing=true and dir is not writable Failure/Error: Unable to find matching line from backtrace Errno::EACCES: Permission denied - /sbuild-nonexistent

./lib/yard/registry.rb:378:in `local_yardoc_file'

 # ./lib/yard/registry.rb:62:in`yardoc_file_for_gem'
 # ./spec/registry_spec.rb:56:in `block (3 levels) in <top (required)>'

2) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid class 'CallMethod('test')' Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:65:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:65:in`block (3 levels) in <top (required)>'

3) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid class 'VSD^#}}' Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:65:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:65:in`block (3 levels) in <top (required)>'

4) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid class 'not.aclass' Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:65:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:65:in`block (3 levels) in <top (required)>'

5) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid class 'self' Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:65:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:65:in`block (3 levels) in <top (required)>'

6) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid superclass '@@INVALID' but it should create the class. Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:72:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:72:in`block (3 levels) in <top (required)>'

7) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid superclass 'hi' but it should create the class. Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:72:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:72:in`block (3 levels) in <top (required)>'

8) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid superclass '$MYCLASS' but it should create the class. Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:72:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:72:in`block (3 levels) in <top (required)>'

9) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError for invalid superclass 'AnotherClass.new' but it should create the class. Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:72:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:72:in`block (3 levels) in <top (required)>'

10) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError if the constant class reference 'class << SomeConstant' does not point to a valid class name Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:83:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:79:in`block (3 levels) in <top (required)>'

11) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError if the constant class reference 'class << SomeConstant' does not point to a valid class name Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:83:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:79:in`block (3 levels) in <top (required)>'

12) YARD::Handlers::Ruby::ClassHandler should raise an UndocumentableError if the constant class reference 'class << SomeConstant' does not point to a valid class name Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/class_handler_spec.rb:83:in`block (4 levels) in <top (required)>'
 # ./spec/handlers/spec_helper.rb:13:in `with_parser'
 # ./spec/handlers/class_handler_spec.rb:79:in`block (3 levels) in <top (required)>'

13) YARD::Handlers::Ruby::MethodHandler should raise an undocumentable error when a method is defined on an object instance Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/method_handler_spec.rb:134:in`block (2 levels) in <top (required)>'

14) YARD::Handlers::Ruby::PrivateConstantHandler should fail if parameter is not String, Symbol or Constant Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/private_constant_handler_spec.rb:17:in`block (2 levels) in <top (required)>'

15) YARD::Handlers::Ruby::PrivateConstantHandler should fail if constant can't be recognized Failure/Error: Unable to find matching line from backtrace expected YARD::Parser::UndocumentableError but nothing was raised

./spec/handlers/spec_helper.rb:7:in `undoc_error'

 # ./spec/handlers/private_constant_handler_spec.rb:22:in`block (2 levels) in <top (required)>'
graaff commented 12 years ago

We currently don't use C extensions for jruby. I think the problem with the spec in question ("should autolink URLs (markdown specific)") is that it is not only markdown, but rdiscount specific, but not marked as such. Since rdiscount is the preferred markdown provider the spec passes for the other ruby implementations, but not for jruby which uses the next-in-line markdown provider which does not autolink.