search

davispuh / ruby-libmspack

Ruby wrapper for libmspack. Library for compressing and decompressing Microsoft compression formats, CAB, CHM, HLP, LIT, KWAJ and SZDD.
The Unlicense
3 stars 4 forks source link

Fails on Windows #1

Closed alexeymorozov closed 4 years ago

alexeymorozov commented 4 years ago

I have an issue using the library on Windows. Should it work on it? May I do anything to make it work?

Failure/Error: require "libmspack” LoadError: Could not open library 'D:/a/fontist/fontist/vendor/bundle/ruby/2.6.0/gems/libmspack-0.1.0/ext/i386-windows/libmspack.dll'

Stacktrace is here https://github.com/fontist/fontist/pull/146/checks?check_run_id=1118323310

davispuh commented 4 years ago

It's been many years since last time I was running this on Windows but it still should work. Worst case you need to rebuild it. That error message means it can't find/load libmspack.dll is it present there at that location? How did you install it?

alexeymorozov commented 4 years ago

Davis, thank you for the response! I've investigated a little more.

Step 1. Yes, the dll exists at this location:

     LoadError:
       Could not open library 'C:/Ruby26-x64/lib/ruby/gems/2.6.0/bundler/gems/ruby-libmspack-f23ac735b67d/ext/i386-windows/libmspack.dll':
     # C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/ffi-1.13.1-x64-mingw32/lib/ffi/library.rb:145:in `block in ffi_lib'
     # C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/ffi-1.13.1-x64-mingw32/lib/ffi/library.rb:99:in `map'
     # C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/ffi-1.13.1-x64-mingw32/lib/ffi/library.rb:99:in `ffi_lib'
     # C:/Ruby26-x64/lib/ruby/gems/2.6.0/bundler/gems/ruby-libmspack-f23ac735b67d/lib/libmspack.rb:18:in `<module:LibMsPack>'
PS C:\vagrant\fontist> dir C:/Ruby26-x64/lib/ruby/gems/2.6.0/bundler/gems/ruby-libmspack-f23ac735b67d/ext/i386-windows/

    Directory: C:\Ruby26-x64\lib\ruby\gems\2.6.0\bundler\gems\ruby-libmspack-f23ac735b67d\ext\i386-windows

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----        9/24/2020   1:08 PM         180168 libmspack.dll

Step 2. Tried to remove the i386 version and use the x86_64 version (ext/x86_64-windows/libmspack.dll). It generates FFI::NullPointerError:

     FFI::NullPointerError:
       invalid memory read at address=0x0000000000000000
     # C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/ffi-1.13.1-x64-mingw32/lib/ffi/struct_layout.rb:88:in `get'
     # C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/ffi-1.13.1-x64-mingw32/lib/ffi/struct_layout.rb:88:in `get'
     # C:/Ruby26-x64/lib/ruby/gems/2.6.0/bundler/gems/ruby-libmspack-e406dd7579d1/lib/libmspack/mscab.rb:305:in `[]'

Step 3. Also tried to recompile versions 1.9.1, 1.7, and already used 1.5 - all generate the same NullPointerError.

Anything else I could try? Would appreciate any help. Thanks

davispuh commented 4 years ago

Could not open library 'C:/Ruby26-x64/lib/ruby/gems/2.6.0/bundler/gems/ruby-libmspack-f23ac735b67d/ext/i386-windows/libmspack.dll':

C:/Ruby26-x64/lib/ruby/gems/2.6.0/gems/ffi-1.13.1-x64-mingw32/lib/ffi/library.rb:145:in `block in ffi_lib'

Hmm, this is looks really strange. So you have 64-bit Ruby and 64-bit FFI but for some reason it tries to load 32-bit libmspack.dll so this is main error, that's why it says can't open library as they're not compatible. So this is first thing you can look into why it thinks it needs to use 32-bit even when you're using 64-bit Ruby.

FFI::NullPointerError: invalid memory read at address=0x0000000000000000

This is separate issue. Might be related to used compiler, maybe if Ruby is compiled with one compiler and library with different one but not sure. It's pretty easy to break ABI with different compilation flags/options but then again I think for FFI shouldn't matter that much... Basically just need to debug this and see if that dll gets loaded and how far you get in it where it crashes, check with GDB.

alexeymorozov commented 4 years ago

Thank you, Davis. I will investigate a little more

davispuh commented 4 years ago

I was able to reproduce this, the problem is here https://github.com/ffi/ffi-compiler/blob/master/lib/ffi-compiler/loader.rb#L12

Dir.glob("#{path}/**/{#{FFI::Platform::ARCH}-#{FFI::Platform::OS}/#{library},#{library}}")

FFI::Platform::ARCH is correct x86_64-windows but looks like Ruby have changed Glob behavior and for some reason fallback is first in list. So we do this glob ruby/gems/2.7.0/gems/libmspack-0.1.0/lib/**/{x86_64-windows/libmspack.dll,libmspack.dll} which returns ["D:/Ruby27-x64/lib/ruby/gems/2.7.0/gems/libmspack-0.1.0/ext/i386-windows/libmspack.dll", "D:/Ruby27-x64/lib/ruby/gems/2.7.0/gems/libmspack-0.1.0/ext/x86_64-windows/libmspack.dll"] Notice how it's in opposite order. I'll fix this in next couple of days. And lol this took me 5mins to find, idk why it was so difficult for you :D

alexeymorozov commented 4 years ago

:D Thank you, Davis!

The behavior of ‘Glob’ is unevident indeed.

I’m glad the issue is finally resolved. Very appreciate the fix.

alexeymorozov commented 4 years ago

Made a PR with the changes https://github.com/davispuh/ffi-compiler/pull/2. Please review, when you have time.

Since a forked version of ffi-compiler2 could not be easily added to a project (tried several ways), I would help with the release as fast as I can.

davispuh commented 4 years ago

It's usually pretty simple to use a fork, like you just need to add it in your Gemfile, see https://bundler.io/v2.1/man/gemfile.5.html#GIT

Anyway I just released v2.0.1 with your fix. Let me know if libmspack now works so this can be closed then.

alexeymorozov commented 4 years ago

I'm very happy that we have a release now :)

Specifying a fork of ffi-compiler was tricky to me. I've even tried to fork ruby-libmspack and change its Gemfile. But the only thing worked was to copy whole ffi-compiler to the forked ruby-libmspack :facepalm:

It works now! Hurray! :)

Davis, I'm very grateful for your help