lsegal / yard

YARD is a Ruby Documentation tool. The Y stands for "Yay!"
http://yardoc.org
MIT License
1.92k stars 394 forks source link

`.yardopts` refuses filenames containing non-ASCII characters #1517

Open noraj opened 8 months ago

noraj commented 8 months ago

.yardopts refuses filenames containing non-ASCII characters

Steps to reproduce

Taking the following file:

module Test
  VERSION = '0.0.1'
end

Name it testé.rb and run yard doc testé.rb --no-yardopts: everything is fine.

Now create a file named .yardopts with the following content

testé.rb

It gives the following error (see Actual Output).

Actual Output

/home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/logging.rb:104:in `block in progress': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/logging.rb:103:in `synchronize'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/logging.rb:103:in `progress'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/logging.rb:81:in `capture'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/parser/source_parser.rb:45:in `parse'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/parser/source_parser.rb:371:in `parse_in_order'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/parser/source_parser.rb:114:in `block in parse'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/logging.rb:182:in `enter_level'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/parser/source_parser.rb:113:in `parse'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard.rb:20:in `parse'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/cli/yardoc.rb:259:in `block in run'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/serializers/yardoc_serializer.rb:56:in `lock_for_writing'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/registry_store.rb:202:in `lock_for_writing'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/registry.rb:210:in `lock_for_writing'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/cli/yardoc.rb:258:in `run'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/cli/command.rb:14:in `run'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/cli/command_parser.rb:72:in `run'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/lib/yard/cli/command_parser.rb:54:in `run'
        from /home/noraj/.asdf/installs/ruby/3.2.2/lib/ruby/gems/3.2.0/gems/yard-0.9.34/bin/yard:13:in `<top (required)>'
        from /home/noraj/.asdf/installs/ruby/3.2.2/bin/yard:25:in `load'
        from /home/noraj/.asdf/installs/ruby/3.2.2/bin/yard:25:in `<main>'

Expected Output

No error.

Environment details:

(Un)related issues

All issues I found for that error are from Ruby 1.9.3 from 2013. But since Ruby 2.0 there is no need to provide an encoding comment to specify the source encoding. Generally there is no issue with encoding with yard even if my files are full of non-ASCII characters in the comments, content and file names. There only issue is when the file name contains non-ASCII chars in .yardopts.

Debug

This where yard seems to crash

https://github.com/lsegal/yard/blob/2d197a381c5d4cc5c55b2c60fff992b31c986361/lib/yard/logging.rb#L104

However, running print("\e[2K\e[?25l\e[1mtesté\e[0m\r") does not seem to crash.

noraj commented 8 months ago

Weird, I pushed this .yardopts file https://github.com/IDLFAC/CyberMots-ILC/blob/a16e3a4f4df91c3a1d4d4e883f873cc616a10034/.yardopts that makes yard crash locally with ruby 3.2.2 and yard 0.9.34 but I gave my github repository to rubydoc.info https://www.rubydoc.info/github/IDLFAC/CyberMots-ILC/ and it successfully built it with the same version Generated on Sun Nov 5 19:25:34 2023 by [yard](http://yardoc.org/) 0.9.34 (ruby-3.2.2)..

Is yard info using a special / patched version of yard? A different configuration? Using last commit from master and not the release? It must take my .yardopts file into consideration because for the gem when there was no .yardopts the documentation was blank https://www.rubydoc.info/gems/cybermots because the library folder is non standard.