jmarrec
commented
4 years ago I managed to compile on MSVC by adding macro definitions to avoid the winsocket redefinition issues.
Closed jmarrec closed 3 years ago
jmarrec
commented
4 years ago I managed to compile on MSVC by adding macro definitions to avoid the winsocket redefinition issues.
flavorjones
commented
4 years ago (CCing @larskanis who know more about Windows than I do and may be able to help.)
Hi, @jmarrec, thanks for opening this issue. It looks like you've jumped to a solution in #2016, which is great, thank you -- but I'd like to have a conversation here to more deeply understand the problem you're trying to solve.
Your comments here, and in the PR, about jaro_winkler seems like a distraction from the core problem that I understand, which is "Does Nokogiri compile under MSVC?". Can you help me understand why you brought it up, and why it's added to the Gemfile in the PR? I'd like to consider jaro_winkler out of scope, but can't until I understand why you're mentioning it.
I don't know what conan is, or why it's brought up here. If you're compiling Nokogiri from source, it's going to assume you have a zlib library installed to link against. If there are issues compiling the vendored libxml2 or libxslt then I'd like to understand what those problems are and fix them, rather than introduce the XY problem of making conan work for you.
If that we can restate the problem without jaro_winkler or conan, I'd like to do that. If that's not possible, I'd like to understand why.
flavorjones
commented
4 years ago In the meantime, I've started the process of downloading and installing a Windows 10 virtualbox VM with VS 2019 to try to reproduce what you might be seeing. That will ... take some time.
jmarrec
commented
4 years ago @flavorjones yeah thanks for getting back to me, and I apologize is this is somewhat messy :)
I'm going to quote something I wrote in #2016 so it's all here:
This fixes the redefinition issues, but ideally the packaged (=vendored) dependencies (zlib, libxml, lixslt) would work out of the box (I am using conan to get the dependencies and passing some arguments such as
--with-xxx-dirto make it work right now)
The vendored libxml2 and libxlst wouldn't compile indeed on MSVC
homebrew on mac would) for these dependencies that would either download pre-built MSVC binaries or help in building it locally.jaro_winkler:
rubocop (since version 0.57.0). rubocop 0.73+ being a hard dependency of nokogiri, jaro_winkler is one of your dependencies. https://github.com/sparklemotion/nokogiri/blob/ace33407faec3d65e07bf536db349d86788257fd/Gemfile#L21So to summarize:
flavorjones
commented
4 years ago @jmarrec Thanks for the additional context.
I'll note that jaro_winkler is only a development dependency, and not a runtime dependency. If you're having to build from source, you can safely remove rubocop from your Gemfile, no big deal.
If you'd like to remove the Gemfile patch from #2016, and let the PR pipeline run, it should go green and I'd be happy to merge it.
I'll update #2016 with this note as well.
MSP-Greg
commented
4 years ago @flavorjones
There is an mswin Ruby master build available as an GitHub release artifact. It's easy to use in Actions, and ruby/openssl is running CI against it at https://github.com/ruby/openssl/runs/497301673
extconf/mkmf is stopping right away when I try to install the gem.
Are you interested in testing with mswin in CI?
jmarrec
commented
4 years ago Added a TODO to the OP above with three items, only one of which is addressed in #2016.
flavorjones
commented
4 years ago @jmarrec Great, thanks. I'm removing the v1.11.0 milestone marker, then, because this is a bit bigger problem than I'm ready to take on right now.
@MSP-Greg Sorry for not responding yet to your other CI-related issues, will circle back on those when I have a moment.
MSP-Greg
commented
4 years ago @flavorjones
Thanks. I know you've used Concourse-CI for a while, and times are different now.
Actions is a bit different, and some of the problems people had with AppVeyor & Travis are now being handled by Ruby people.
larskanis
commented
4 years ago I successfully mix MSVC compiled binaries into MINGW code in several projects and it should work the other way around as well. That's what I usually recommend users, when a library isn't available/compatible for the particular platform.
Regarding MSVC support in nokogiri I can not really help - I don't use MSVC and don't have any interest in doing so. But sure, I'll keep maintaining the MINGW port.
flavorjones
commented
3 years ago I'd like to close this, as I'm not sure what remains to be done and I am unaware of any ongoing work in this area.
If MSVC still does not compile Nokogiri, I'm open to a PR that addresses the issue, but I won't be doing that work myself and so I'd like to close this issue or find an owner who's willing to commit to defining and finishing the work. I'd expect setting up CI for MSVC to be part of that work, both to demonstrate the problems and to demonstrate that they've been solved.
KoellM
commented
3 years ago 1.11.0 and 1.11.1 can not be installed on MSVC. This is a patch:
# ext/nokogiri/extconf.rb
# Before
# ensure_package_configuration(opt: "zlib", pc: "zlib", lib: "z", headers: "zlib.h", func: "gzdopen")
# After
# The change maybe affect other platforms, I’m not sure
ensure_package_configuration(opt: "zlib", pc: "zlib", lib: "zlib", headers: "zlib.h", func: "gzdopen")OK, I'm going to close this for the reasons I've mentioned above.
Once again, I'm open to PRs that address specific MSVC issues, but then I would like these each treated as separate, addressable issues -- using the bug report or installation problem templates -- so that we understand the failure modes and can have a conversation about how to fix it.
What I really want is someone to help set up CI of some kind on MSVC so that we have a feedback loop on proposed and actual changes. This is an opportunity for people who depend on MSVC to show up in the community and contribute.
@KoellM if you'd like to have a conversation about the failure you're trying to address with the suggested patch, can you please open a new "installation difficulties" issue, and include all the necessary logs from the failed install? Thanks.
MSP-Greg
commented
3 years ago What I really want is someone to help set up CI of some kind on MSVC
Testing against Ruby mswin master can be done with GitHub Actions. ruby/openssl has been doing it for quite a while, see:
flavorjones
commented
3 years ago @MSP-Greg thanks once again for being a voice in support of Github Actions. If you want to submit a PR introducing MSVC test into the CI suite (is that the same as mswin? I honestly have no idea) then I'm open to it, especially if you're a Windows user.
flavorjones
commented
3 years ago @MSP-Greg maybe here I should also qualify my remarks: I'd like some way to validate that CI is reproducing what MSVC users are doing in the real world. Which is why I keep asking Windows users to do this work.
I want to avoid the antipattern of spending time setting up a red build, then spending time getting red to green, and then discovering that real-world cases are actually harder/simpler/different. How can we mitigate that risk?
TODO
.cross-rubiesandtask/cross-ruby.rbwould be modified to allow MSVC gems to be packaged.Describe the bug
I found the similar https://github.com/sparklemotion/nokogiri/issues/255 which is from 2010 and seemed to imply MSVC support was achieved, yet it isn't.
I realize this is rather a corner case, but it's one I am in: I have to use MSVC to compile this gem as we have a C++ application that has to be built on MSVC and one thing it does is to create a CLI based on ruby and embedded some gems.
To Reproduce
bundle installfailed.gem 'jaro_winkler', '= 1.5.4', :github => 'jmarrec/jaro_winkler', :ref => 'f1ca425fdef06603e5c65b09c5b681f805e1e297'toGemfilebundle installworksbundle exec rake compilewill fail right away due to the unability to compile the packaged libs (zlib and co) with the currentextconf.rbconanto download the dependencies (zlib, xml2, and xlst/exlst)I created a
conanfile.txtlike thisInstall dependencies and activate the build env
Realized it wouldn't still find the exslt lib due to the naming of the libs, so just cp them to a new name
Ok, try compilation again:
Output:
Expected behavior
Ideally Nokogiri would built out of the box on MSVC.
Environment
Windows 10 x64. Ruby compiled locally with MSVC 2019 16.5
$ ruby -v ruby 2.5.5p157 (2019-03-15) [x64-mswin64_140]
$ gem list
LOCAL GEMS
ast (2.4.0) bigdecimal (default: 1.3.4) bundle (0.0.1) bundler (2.1.4) cmath (default: 1.0.0) concourse (0.31.0) csv (default: 1.0.0) date (default: 1.0.0) docile (1.3.2) fiddle (default: 1.0.0) fileutils (default: 1.0.2) hoe (3.22.1) hoe-bundler (1.5.0) hoe-debugging (2.0.0) hoe-gemspec (1.0.0) hoe-git (1.6.0) io-console (default: 0.4.6) ipaddr (default: 1.2.0) json (default: 2.1.0) mini_portile2 (2.5.0) minitest (5.14.0) openssl (default: 2.1.2) parallel (1.19.1) parser (2.7.0.5) psych (default: 3.0.2) racc (1.4.16) rainbow (3.0.0) rake (13.0.1) rake-compiler (1.1.0) rake-compiler-dock (1.0.1) rdoc (6.2.1, default: 6.0.1) rexical (1.0.7) rexml (3.2.4) rubocop (0.80.1) ruby-progressbar (1.10.1) scanf (default: 1.0.0) simplecov (0.18.5) simplecov-html (0.12.2) sync (0.5.0) term-ansicolor (1.7.1) tins (1.24.1) unicode-display_width (1.6.1) webrick (default: 1.4.2)