search

kaitai-io / kaitai_struct_visualizer

Kaitai Struct: visualizer and hex viewer tool
https://rubygems.org/gems/kaitai-struct-visualizer
GNU General Public License v3.0
280 stars 25 forks source link

Find out what Ruby versions ksv is compatible with on Windows #51

Closed generalmimon closed 1 year ago

generalmimon commented 2 years ago

Ruby installations come from https://rubyinstaller.org/downloads/, everything is run in Windows Sandbox. kaitai-struct-visualizer-0.7.gem was built manually from https://github.com/kaitai-io/kaitai_struct_visualizer/pull/49 by using gem build kaitai-struct-visualizer.gemspec with the humdogm:master branch checked out.

Ruby 1.8

C:\Users\WDAGUtilityAccount>ruby --version
ruby 1.8.7 (2013-06-27 patchlevel 374) [i386-mingw32]

C:\Users\WDAGUtilityAccount>gem --version
1.8.24

C:\Users\WDAGUtilityAccount>gem install kaitai-struct-visualizer-0.7.gem
Fetching: kaitai-struct-0.10.gem (100%)
Successfully installed kaitai-struct-0.10
Successfully installed kaitai-struct-visualizer-0.7
2 gems installed
Installing ri documentation for kaitai-struct-0.10...
Installing ri documentation for kaitai-struct-visualizer-0.7...
Installing RDoc documentation for kaitai-struct-0.10...
Installing RDoc documentation for kaitai-struct-visualizer-0.7...

C:\Users\WDAGUtilityAccount>ksv --help
C:/Ruby187/lib/ruby/gems/1.8/gems/kaitai-struct-visualizer-0.7/bin/ksv:9: undefined method `realpath' for File:Class (NoMethodError)
        from C:/Ruby187/bin/ksv:23:in `load'
        from C:/Ruby187/bin/ksv:23

https://github.com/kaitai-io/kaitai_struct_visualizer/blob/5b3ec3dbaa6f3cbc9c34c0e51d92d443c3861e71/bin/ksv#L9

Ruby 1.9

C:\Users\WDAGUtilityAccount>ruby --version
ruby 1.9.3p551 (2014-11-13) [i386-mingw32]

C:\Users\WDAGUtilityAccount>gem --version
1.8.29

C:\Users\WDAGUtilityAccount>gem install kaitai-struct-visualizer-0.7.gem
Fetching: kaitai-struct-0.10.gem (100%)
Successfully installed kaitai-struct-0.10
Successfully installed kaitai-struct-visualizer-0.7
2 gems installed
Installing ri documentation for kaitai-struct-0.10...
Installing ri documentation for kaitai-struct-visualizer-0.7...

RDoc::Parser::Ruby failure around line 16 of
lib/kaitai/tui.rb

Before reporting this, could you check that the file you're documenting
has proper syntax:

  C:/Ruby193/bin/ruby.exe -c lib/kaitai/tui.rb

RDoc is not a full Ruby parser and will fail when fed invalid ruby programs.

The internal error was:

        (RDoc::Error) unknown type of %string "i"

ERROR:  While generating documentation for kaitai-struct-visualizer-0.7
... MESSAGE:   unknown type of %string "i"
... RDOC args: --ri --op C:/Ruby193/lib/ruby/gems/1.9.1/doc/kaitai-struct-visualizer-0.7/ri lib --title kaitai-struct-visualizer-0.7 Documentation --quiet

C:\Users\WDAGUtilityAccount>ruby -c ksv-src\lib\kaitai\tui.rb
ksv-src/lib/kaitai/tui.rb:16: unknown type of %string
        @highlight_colors = %i[white aqua blue green white]
                               ^
ksv-src/lib/kaitai/tui.rb:16: syntax error, unexpected $end
        @highlight_colors = %i[white aqua blue green white]
                               ^

https://github.com/kaitai-io/kaitai_struct_visualizer/blob/5b3ec3dbaa6f3cbc9c34c0e51d92d443c3861e71/lib/kaitai/tui.rb#L16

Ruby 2.0

C:\Users\WDAGUtilityAccount>ruby --version
ruby 2.0.0p648 (2015-12-16) [i386-mingw32]

C:\Users\WDAGUtilityAccount>gem --version
2.0.14.1

C:\Users\WDAGUtilityAccount>gem install kaitai-struct-visualizer-0.7.gem
ERROR:  While executing gem ... (Gem::DependencyError)
    Unable to resolve dependencies: kaitai-struct-visualizer requires kaitai-struct (~> 0.4)

C:\Users\WDAGUtilityAccount>gem install kaitai-struct
ERROR:  Could not find a valid gem 'kaitai-struct' (>= 0), here is why:
          Unable to download data from https://rubygems.org/ - SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (https://rubygems.org/latest_specs.4.8.gz)

C:\Users\WDAGUtilityAccount>gem install kaitai-struct-0.10.gem
Successfully installed kaitai-struct-0.10
Parsing documentation for kaitai-struct-0.10
Installing ri documentation for kaitai-struct-0.10
1 gem installed

C:\Users\WDAGUtilityAccount>gem install kaitai-struct-visualizer-0.7.gem
Successfully installed kaitai-struct-visualizer-0.7
Parsing documentation for kaitai-struct-visualizer-0.7
Installing ri documentation for kaitai-struct-visualizer-0.7
WARNING:  Unable to pull data from 'https://rubygems.org/': SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (https://rubygems.org/specs.4.8.gz)
1 gem installed

C:\Users\WDAGUtilityAccount>ksv --help
C:/Ruby200/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require': C:/Ruby200/lib/ruby/gems/2.0.0/gems/kaitai-struct-visualizer-0.7/lib/kaitai/struct/visualizer/hex_viewer.rb:128: syntax error, unexpected '.' (SyntaxError)
          @tree&.do_exit
                 ^
    from C:/Ruby200/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from C:/Ruby200/lib/ruby/gems/2.0.0/gems/kaitai-struct-visualizer-0.7/lib/kaitai/struct/visualizer/tree.rb:7:in `<top (required)>'
    from C:/Ruby200/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from C:/Ruby200/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from C:/Ruby200/lib/ruby/gems/2.0.0/gems/kaitai-struct-visualizer-0.7/lib/kaitai/struct/visualizer/parser.rb:5:in `<top (required)>'
    ...

C:\Users\WDAGUtilityAccount>ruby -c C:/Ruby200/lib/ruby/gems/2.0.0/gems/kaitai-struct-visualizer-0.7/lib/kaitai/struct/visualizer/hex_viewer.rb
C:/Ruby200/lib/ruby/gems/2.0.0/gems/kaitai-struct-visualizer-0.7/lib/kaitai/struct/visualizer/hex_viewer.rb:128: syntax error, unexpected '.'
          @tree&.do_exit
                 ^

https://github.com/kaitai-io/kaitai_struct_visualizer/blob/5b3ec3dbaa6f3cbc9c34c0e51d92d443c3861e71/lib/kaitai/struct/visualizer/hex_viewer.rb#L128

https://stackoverflow.com/a/36812667/12940655:

It is called the Safe Navigation Operator. Introduced in Ruby 2.3.0, it lets you call methods on objects without worrying that the object may be nil(Avoiding an undefined method for nil:NilClass error), similar to the try method in Rails.

Ruby 2.3

C:\Users\WDAGUtilityAccount>ruby --version
ruby 2.3.3p222 (2016-11-21 revision 56859) [x64-mingw32]

C:\Users\WDAGUtilityAccount>gem --version
2.5.2

C:\Users\WDAGUtilityAccount>gem install kaitai-struct-visualizer-0.7.gem
Successfully installed kaitai-struct-0.10
Successfully installed kaitai-struct-visualizer-0.7
Parsing documentation for kaitai-struct-0.10
Installing ri documentation for kaitai-struct-0.10
Parsing documentation for kaitai-struct-visualizer-0.7
Installing ri documentation for kaitai-struct-visualizer-0.7
Done installing documentation for kaitai-struct, kaitai-struct-visualizer after 0 seconds
WARNING:  Unable to pull data from 'https://rubygems.org/': SSL_connect returned=1 errno=0 state=error: certificate verify failed (https://api.rubygems.org/specs.4.8.gz)
2 gems installed

C:\Users\WDAGUtilityAccount>ksv --help
Usage: ksv [options] <file_to_parse.bin> <format.ksy>...|<format.rb>

    -I, --import-path [DIRECTORIES]  .ksy library search path(s) for imports (see also KSPATH env variable)
        --opaque-types [BOOLEAN]     opaque types allowed, default: false
    -r, --require [PATH]             load ("require") .rb file into Ruby process
        --version                    show versions of ksv and ksc

C:\Users\WDAGUtilityAccount>ksv example.cdr.unpk cdr_unpk.ksy
Compilation OK
... processing cdr_unpk.ksy 0
...... loading file_streams.rb
...... loading coreldraw_cdr.rb
...... loading cdr_unpk.rb
Classes loaded OK, main class = CdrUnpk
C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/kaitai-struct-visualizer-0.7/lib/kaitai/console_windows.rb:97:in `goto': undefined method `unpack1' for "\x00\x00\x00\x00":String (NoMethodError)
Did you mean?  unpack
    from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/kaitai-struct-visualizer-0.7/lib/kaitai/console_windows.rb:92:in `clear'
    from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/kaitai-struct-visualizer-0.7/lib/kaitai/struct/visualizer/tree.rb:163:in `block in redraw'
    from C:/Ruby23-x64/lib/ruby/2.3.0/benchmark.rb:308:in `realtime'
    from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/kaitai-struct-visualizer-0.7/lib/kaitai/struct/visualizer/tree.rb:162:in `redraw'
    from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/kaitai-struct-visualizer-0.7/lib/kaitai/struct/visualizer/visualizer.rb:20:in `run'
    from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/kaitai-struct-visualizer-0.7/bin/ksv:59:in `<top (required)>'
    from C:/Ruby23-x64/bin/ksv:22:in `load'
    from C:/Ruby23-x64/bin/ksv:22:in `<main>'

I was actually trying out https://github.com/kaitai-io/kaitai_struct_visualizer/pull/49, so line 97 is the following:

https://github.com/kaitai-io/kaitai_struct_visualizer/blob/2e19269aa37c017987108758fbd921eb57f961cd/lib/kaitai/console_windows.rb#L97

And I already know that String#unpack1 was introduced in Ruby 2.4: https://github.com/kaitai-io/kaitai_struct_ruby_runtime/issues/8

Ruby 2.4

C:\Users\WDAGUtilityAccount>ruby --version
ruby 2.4.10p364 (2020-03-31 revision 67879) [x64-mingw32]

C:\Users\WDAGUtilityAccount>gem --version
2.6.14.4

C:\Users\WDAGUtilityAccount>gem install kaitai-struct-visualizer-0.7.gem
Successfully installed kaitai-struct-0.10
Successfully installed kaitai-struct-visualizer-0.7
Parsing documentation for kaitai-struct-0.10
Installing ri documentation for kaitai-struct-0.10
Parsing documentation for kaitai-struct-visualizer-0.7
Installing ri documentation for kaitai-struct-visualizer-0.7
Done installing documentation for kaitai-struct, kaitai-struct-visualizer after 0 seconds
2 gems installed

C:\Users\WDAGUtilityAccount>ksv --help
Usage: ksv [options] <file_to_parse.bin> <format.ksy>...|<format.rb>

    -I, --import-path [DIRECTORIES]  .ksy library search path(s) for imports (see also KSPATH env variable)
        --opaque-types [BOOLEAN]     opaque types allowed, default: false
    -r, --require [PATH]             load ("require") .rb file into Ruby process
        --version                    show versions of ksv and ksc

C:\Users\WDAGUtilityAccount>ksv 8f.bin hello_world.ksy

It seems to work fine (with changes from https://github.com/kaitai-io/kaitai_struct_visualizer/pull/49), the hex viewer finally shows up.

meta:
  id: hello_world
seq:
  - id: one
    type: u1
[-] [root]                                 00000000: 8f                                              | .
  [.] one = 143

Ruby 2.7

C:\Users\WDAGUtilityAccount>ruby --version
ruby 2.7.6p219 (2022-04-12 revision c9c2245c0a) [x64-mingw32]

C:\Users\WDAGUtilityAccount>gem --version
3.1.6

C:\Users\WDAGUtilityAccount>gem install kaitai-struct-visualizer-0.7.gem
Successfully installed kaitai-struct-0.10
Successfully installed kaitai-struct-visualizer-0.7
Parsing documentation for kaitai-struct-0.10
Installing ri documentation for kaitai-struct-0.10
Parsing documentation for kaitai-struct-visualizer-0.7
Installing ri documentation for kaitai-struct-visualizer-0.7
Done installing documentation for kaitai-struct, kaitai-struct-visualizer after 0 seconds
2 gems installed

C:\Users\WDAGUtilityAccount>ksv --help
Usage: ksv [options] <file_to_parse.bin> <format.ksy>...|<format.rb>

    -I, --import-path [DIRECTORIES]  .ksy library search path(s) for imports (see also KSPATH env variable)
        --opaque-types [BOOLEAN]     opaque types allowed, default: false
    -r, --require [PATH]             load ("require") .rb file into Ruby process
        --version                    show versions of ksv and ksc

C:\Users\WDAGUtilityAccount>ksv 8f.bin hello_world.ksy

Works the same as Ruby 2.4.

Ruby 3.0

For Ruby 3.0+ support on Windows (since the version from RubyGems fails with in `require': cannot load such file -- Win32API (LoadError) on line lib/kaitai/console_windows.rb:2 since Ruby 3 - see https://github.com/kaitai-io/kaitai_struct_visualizer/issues/48), I checked out https://github.com/kaitai-io/kaitai_struct_visualizer/pull/49 and built kaitai-struct-visualizer-0.7.gem from sources using gem build kaitai-struct-visualizer.gemspec.

C:\Users\WDAGUtilityAccount>ruby --version
ruby 3.0.4p208 (2022-04-12 revision 3fa771dded) [x64-mingw32]

C:\Users\WDAGUtilityAccount>gem --version
3.2.33

C:\Users\WDAGUtilityAccount>gem install kaitai-struct-visualizer-0.7.gem
Successfully installed kaitai-struct-0.10
Successfully installed kaitai-struct-visualizer-0.7
Parsing documentation for kaitai-struct-0.10
Installing ri documentation for kaitai-struct-0.10
Parsing documentation for kaitai-struct-visualizer-0.7
Installing ri documentation for kaitai-struct-visualizer-0.7
Done installing documentation for kaitai-struct, kaitai-struct-visualizer after 0 seconds
2 gems installed

C:\Users\WDAGUtilityAccount>ksv 8f.bin hello_world.ksy

Works, same as Ruby 2.4 and 2.7.

Ruby 3.1

C:\Users\WDAGUtilityAccount>ruby --version
ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x64-mingw-ucrt]

C:\Users\WDAGUtilityAccount>gem --version
3.3.7

C:\Users\WDAGUtilityAccount>gem install kaitai-struct-visualizer-0.7.gem
Fetching kaitai-struct-0.10.gem
Successfully installed kaitai-struct-0.10
Successfully installed kaitai-struct-visualizer-0.7
Parsing documentation for kaitai-struct-0.10
Installing ri documentation for kaitai-struct-0.10
Parsing documentation for kaitai-struct-visualizer-0.7
Installing ri documentation for kaitai-struct-visualizer-0.7
Done installing documentation for kaitai-struct, kaitai-struct-visualizer after 0 seconds
2 gems installed

C:\Users\WDAGUtilityAccount>ksv 8f.bin hello_world.ksy

Same as Ruby 2.4, 2.7 and 3.0.

generalmimon commented 2 years ago

The reasons why I did this were:

  1. the declared "required Ruby version" at https://rubygems.org/gems/kaitai-struct-visualizer is >= 0:

    REQUIRED RUBY VERSION on `kaitai-struct-visualizer` package at RubyGems

    which is not really true as I discovered - so I think it makes sense to declare it in kaitai-struct-visualizer.gemspec so that it'll be correct on RubyGems as well after a new release.

    @GreyCat also mentioned in https://github.com/kaitai-io/kaitai_struct_visualizer/pull/34#issuecomment-590288856 that:

    Ideally, we'd like to have a clear understanding which versions of Ruby we target here, but this is a good step forward :)

  2. I wanted to test https://github.com/kaitai-io/kaitai_struct_visualizer/pull/49 properly to determine whether it fixes the Ruby 3 incompatibility on Windows while not breaking older versions of Ruby.

    Also I wanted to know how ready we are to publish a new version of ksv to RubyGems.

generalmimon commented 1 year ago

Ruby 2.4 is now specified in .gemspec as the minimum supported Ruby version:

https://github.com/kaitai-io/kaitai_struct_visualizer/blob/347a146d9ea684da558834153216ba950703293f/kaitai-struct-visualizer.gemspec#L25

so this issue can be closed.