Unicode characters in the Ruby or README files is breaking YARD with incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError
When I used the superscript 3 character (³) in my files, YARD will fail to generate correct output.
I have verified that I saved the files with UTF-8 encoding by inspecting the files with a hex-editor:
C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/templates/default/layout/html/layout.erb:21:in `_erb_cache_5': incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError)
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:289:in `erb'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/templates/default/layout/html/setup.rb:62:in `layout'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:369:in `render_section'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:261:in `block (2 levels) in run'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:258:in `each'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:258:in `block in run'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:400:in `add_options'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:257:in `run'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:136:in `run'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/templates/default/fulldoc/html/setup.rb:52:in `block in serialize_index'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/engine.rb:123:in `block in with_serializer'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/logging.rb:82:in `capture'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/engine.rb:121:in `with_serializer'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/templates/default/fulldoc/html/setup.rb:51:in `serialize_index'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/templates/default/fulldoc/html/setup.rb:68:in `serialize_file'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/templates/default/fulldoc/html/setup.rb:11:in `block in init'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/templates/default/fulldoc/html/setup.rb:10:in `each'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/templates/default/fulldoc/html/setup.rb:10:in `each_with_index'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/templates/default/fulldoc/html/setup.rb:10:in `init'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:195:in `initialize'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:131:in `new'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:136:in `run'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/engine.rb:105:in `generate'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/cli/yardoc.rb:357:in `run_generate'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/cli/yardoc.rb:267:in `run'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/cli/command.rb:14:in `run'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/cli/command_parser.rb:72:in `run'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/cli/command_parser.rb:54:in `run'
from C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/bin/yard:13:in `<top (required)>'
from C:/Ruby32-x64/bin/yard:32:in `load'
from C:/Ruby32-x64/bin/yard:32:in `<main>'
If I remove the superscript 3 from the README the output becomes:
C:\Users\tthomas2\Desktop\yard-bug>yard doc --debug
[debug]: Parsing ["*.rb"] with `ruby` parser
[debug]: Parsing hello.rb
[debug]: Serializing to .yardoc/objects/root.dat
[debug]: Re-generating object ...
[debug]: Re-generating object Hello...
[debug]: Generating asset js/jquery.js
[debug]: Serializing to doc/js/jquery.js
[debug]: Generating asset js/app.js
[debug]: Serializing to doc/js/app.js
[debug]: Generating asset js/full_list.js
[debug]: Serializing to doc/js/full_list.js
[debug]: Generating asset css/style.css
[debug]: Serializing to doc/css/style.css
[debug]: Generating asset css/common.css
[debug]: Serializing to doc/css/common.css
[debug]: Generating asset css/full_list.css
[debug]: Serializing to doc/css/full_list.css
[debug]: Generating asset class_list.html
[debug]: Serializing to doc/class_list.html
[debug]: Generating asset method_list.html
[debug]: Serializing to doc/method_list.html
[debug]: Generating asset file_list.html
[debug]: Serializing to doc/file_list.html
[debug]: Generating asset frames.html
[debug]: Serializing to doc/frames.html
[debug]: Serializing to doc/_index.html
[debug]: Serializing to doc/index.html
[debug]: Serializing to doc/file.README.html
[debug]: Serializing to doc/top-level-namespace.html
[error]: Exception occurred while generating 'Hello.html'
[error]: CompatibilityError: incompatible character encodings: ASCII-8BIT and UTF-8
[error]: Stack trace:
C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/templates/default/layout/html/layout.erb:21:in `_erb_cache_5'
C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:289:in `erb'
C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/templates/default/layout/html/setup.rb:62:in `layout'
C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:369:in `render_section'
C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:261:in `block (2 levels) in run'
C:/Users/tthomas2/.gem/ruby/3.2.0/gems/yard-0.9.34/lib/yard/templates/template.rb:258:in `each'
Files: 1
Modules: 1 ( 1 undocumented)
Classes: 0 ( 0 undocumented)
Constants: 0 ( 0 undocumented)
Attributes: 0 ( 0 undocumented)
Methods: 1 ( 0 undocumented)
50.00% documented
The documentation "completes" but with an inline error message that doesn't stop the process. This time it's the superscript 3 in hello.rb that breaks.
I'm wondering if YARD is loading text from templates as ASCII and thus failing to combine UTF-8 and ASCII strings.
Expected Output
Expected YARD to generate the documentation without errors.
Environment details:
OS: Windows 10
Ruby version (ruby -v): ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x64-mingw-ucrt]
Unicode characters in the Ruby or README files is breaking YARD with
incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError
When I used the superscript 3 character (³) in my files, YARD will fail to generate correct output.
I have verified that I saved the files with UTF-8 encoding by inspecting the files with a hex-editor:![image](https://github.com/lsegal/yard/assets/192418/25b92458-0ef8-442b-9c7b-27ad713a306c)
Steps to reproduce
.yardopts
hello.rb
README.md
Actual Output
If I remove the superscript 3 from the README the output becomes:
The documentation "completes" but with an inline error message that doesn't stop the process. This time it's the superscript 3 in
hello.rb
that breaks.I'm wondering if YARD is loading text from templates as ASCII and thus failing to combine UTF-8 and ASCII strings.
Expected Output
Expected YARD to generate the documentation without errors.
Environment details:
ruby -v
): ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x64-mingw-ucrt]yard -v
): yard 0.9.34I have read the Contributing Guide.