GCC 7.1.0 - problems

[MSP-Greg]

I just updated to GCC 7.1.0. Others may have noticed, but things are not good.

Running tests using

ruby runner.rb -j3 -Ilib --show-skip ruby

Results -

Last GCC 6.3 build I have:

5279 tests, 4110750 assertions, 3 failures, 0 errors, 25 skips
ruby -v: ruby 2.5.0dev (2017-07-03 trunk 59255) [x64-mingw32]

RI2 snapshot (7.1.0):

5279 tests, 3681817 assertions, 69 failures, 4 errors, 25 skips
ruby -v: ruby 2.5.0dev (2017-07-05 trunk 59268) [x64-mingw32]

My build today (7.1.0):

5279 tests, 3680486 assertions, 69 failures, 4 errors, 25 skips
ruby -v: ruby 2.5.0dev (2017-07-06 trunk 59273) [x64-mingw32]

The three 6.3.0 failures are expected outside of the normal make test-all env. I'll try to investigate later today. Many of the failures seem sprintf related.

Edit: if you prefer to revert to 6.3, below are two scripts that should remove and add. Place in the usr/bin folder and run (remove 1st). I had to add the txt extension to attach them...

pkg_add_6.3.0.txt pkg_remove_7.1.0.txt

[larskanis]

This is interesting! I currently run only the basic tests regularly and they didn't show any failures: https://github.com/oneclick/rubyinstaller2-packages/blob/master/mingw-w64-ruby25/PKGBUILD#L72 . I also inspected the gcc release notes for 7.1 before the latest RubyInstaller release.

Can you upload your test output please?

[MSP-Greg]

@larskanis

Sorry for the delay, I'm -0500.

Attached are results from console for test ruby and mspec. As you probably recall, test result I've had for gcc 6x are here.

I used a recent RI2 snapshot build. I've reverted my MSYS2 to 6.3, so I can't test one of my builds, but as mentioned, when I noticed the issue, the test ruby results were identical to your build.

Background info for readers unfamiliar with Ruby testing --

Ruby CI tests on travis, on appveyor.

More Ruby test results are at rubyci.org. Both Ruby appveyor and rubyci.org use VS2013 / 120 builds. So, even though most of the world uses MinGW builds on Windows, the Ruby team tests against VS builds.

Back to testing.

test-all is the most thorough, and takes the most time. It is normally run within a make/nmake statement, which provides functionality for some tests. These tests are located in the test folder in the Ruby source/repo. These tests can be run outside of the make environment, but more failures/errors may occur.

test-rubyspec or mspec tests are designed to be usable on any Ruby platform, but are less thorough than test-all. They are located in the spec/rubyspec folder, and can be run outside of make/nmake.

RI2_snapshot_ruby_7.1.0.txt RI2_snapshot_mspec_7.1.0.txt

[MSP-Greg]

@larskanis

You probably already know this, but Appveyor seems to be on gcc 6.3

[vnluc]

A little confusing because I am not expert. I look at this file Appveyor and look like it use VS/nmake.

from rubyinstaller2 [output](https://ci.appveyor.com/project/larskanis/rubyinstaller2-hbuor/build/job/tadpgxg6alhgkum3) seems it use gcc 7.1.0 but it does not compile any source.

I am currently using "rubyinstaller-2.5.0-snapshot-x64" from Appveyor for my daily script and got no issues (I not running the check).

[MSP-Greg]

@vnluc

I believe I just saw your post at ruby-lang.org. I've got builds at BinTray, along with the three packages I use to build. 64 bit only. Whatever the last z_logs file is matches the DATE/SVN of the ruby_trunk.7z file. All are signed. I set it up so it could be used with appveyor. Due to unresolved issues with gcc 7.1.0, I'm still building wiht 6.3.0.

Using trunk with gem extensions may require editing files or manually configuring for gem install to work. Bundler may work, or may also require configuration. Have at it...

[vnluc]

@MSP-Greg : Thank you for your information, I will try to use your build. But can we download the build output from the ruby ci servers? They do not allow?

[larskanis]

I opened a bug report with a patch for this issue: https://bugs.ruby-lang.org/issues/13748

[MSP-Greg]

@larskanis

Wonderful & thanks. That was what I hoped to look at this weekend, but I doubt I would've figured that one out. As an aside, do you think going down the rabbit hole of using a newer version of msvcr would be something to consider?

From ruby-lang.org

I'm currently unable to run a full "make test-all"

After doing the above:

1. Maybe it's time for me to revisit all my patches (most are for test & spec), update my repo, and we can determine what should be pushed to RI2. I've got some patches for things that are opened at ruby-lang.org but not yet accepted. Sub second file times and another older one. I'm running all tests parallel (-j3) on a quad core, there may be issues doing so on the dual core Appveyor machines.

2. I still let three failures thru, one's from my use of readline, two are odd. One with RubyGems (I think it uses TSort), and one with a test that passes when run in a pure windows env. I think it has to do with what command processor is chosen, and I think I came across code that uses the *nix version if found, regaardless of OS. We can (obviously) patch around these three/two failures.

3. I did open an issue at Alexpux/MINGW-packages about adding ruby specific packages for gdbm, libyaml, & openssl. As mentioned prev, I can not get anything past gdbm 1.10 to test as a package. Not being a c type, I'm not sure I've addressed the issues well. I also compiled them with 6.3.0, I should try a 7.1.0. Packages are at BinTray.

[MSP-Greg]

I just tried building the ruby_2_4 stable branch with everything identical to my normal trunk build.

test-all - five failures, two match 2.5/trunk failures, the third 2.5/trunk failure passed, and there were three failures that appear encoding related. No errors, skips about the same.

Install crashed & burned on on an issue that may match one reported in trunk (bundled gem install), probably not backported. So, results are good. btest and test-basic passed. I didn't run spec tests, I'll try to add them. As you may recall, they were added to the Ruby repo in 2.5, previously they were separate.

I'll try running 2.4.1 later. Thanks again for the patch...

[larskanis]

Thank you for your nice summary to ruby-testing above!

Regarding the tests, I believe we need proper appveyor-tests bounded directly to the ruby repository or at least an integration into http://rubyci.org . This way ruby core developers will have an eye on MINGW specific issues, rather then depending on us to report these. Is this something you're working on?

As an aside, do you think going down the rabbit hole of using a newer version of msvcr would be something to consider?

I'll not use MS Visual C. This is against my hacker ethics and I doubt, that it will make anything better :smirk:

I did open an issue at Alexpux/MINGW-packages about adding ruby specific packages for gdbm, libyaml, & openssl.

I doubt that they will accept this, but it is not even necessary - that's one reason why I created https://github.com/oneclick/rubyinstaller2-packages . We can place all required MINGW packages there and use our own repository on bintray. It can easily integrated into MSYS2 as described in the readme. Maybe the bintray repository can be moved to the oneclick account in the future, but I don't have access to it currently.

[MSP-Greg]

First of all, I started a 2.4.1 build before retiring last night. Six failures, same as 2.4.2/ruby_2_4, except with the addition of the third test that fails in trunk. btest & test-basic passed. I'd like to get spec working locally, but it's messy, as it's in two other repos for ruby 2.4 and earlier.

Thank you for your nice summary to ruby-testing above!

Well, you're being nice referring to it as a 'summary'. With the whole MinGW thing, I'm at times frustrated, and that may cloud some of my posts, especially for someone who isn't a native english speaker (I've been told my cynicism is often subtle.) My apologies to anyone affected.

The test explanation was for those who may not realize what is going on with RI2/etc:

1. It is very new
2. It makes use of loosely coupled OSS build tools (RI was tightly coupled). This allows much less maintainance by the RI2 team.
3. It is laying a foundation that can be built upon and extended by users/organization that have particular requirements. For example, dependency/version/package metadata used by bundler/rubygems.
4. It may have already affected how windows/mingw is viewed by the ruby core team.

But, because it is new, and many of us don't have the time/resources/desire to set up a VM or separate 'fresh' system, issues with extension gems are difficult, especially when we've not had issues with our own installation.

I'll not use MS Visual C

I don't keep track of it, but it was my impression that VS 2015 (or starting with) had a much better license in terms of OSS use. OpenSSL / OpenSSL tests against it. Maybe consider a long term future goal.

Is this something you're working on?

Nothing would make me happier than to have everything building/testing on Appveyor.

Back to immediate work to be done

General

5. Ruby 2.3 - Assuming we want RI2 to gain momentum, and given that many people (gem teams) are probably working with (or would like to work with) both 2.3 and 2.4, consideration should be given to releasing an RI2 2.3 build, and also getting correct builds on Appveyor. I'm sure there's some people frustrated by using RI 2.3 and RI2 2.4. Also, most (all?) 'gemified' default extension gems are installable on 2.3. I haven't looked around, is there a release of 2.3.4 done with gcc 7.1.0? If not, I'll see if I can build one with my fork. I'll run it thru tests...

Testing

6. As mentioned in Misc #13747, determining whether a build should be published is somewhat a grey area. Take the example of the one test mentioned - with msys64/usr/bin in the path (make test), it fails in 2.4.1 and trunk, passes in ruby_2_4 stable. In trunk, it passes without msys64/usr/bin in the path, probably the same for 2.4.1. Anyway, this is less of a problem for releasing version builds, but stable and trunk are a bit different...

7. My current system uses ruby code to apply patches. My patches could be catgorized three ways - build patches, test patches that fix segfaults, and general test patches. They're in three folders, a base folder, a 64 folder, and a 32 folder. 64 and 32 are applied based on the obvious. I laid that out while working on trunk. After that, I determined that patches may also need to be selected based on ruby version. I thought about it, but never decided how best to implement. So, I'm building with shared patches at present, and I did build 2.3 somewhat recently, so shared seems to work for 2.3, 2.4, and trunk. At present, RI2 is using separate patch folders for different ruby builds. There's pluses/minuses for either shared or separate patch files. And we'll be adding 2.6 sometime soon. Your thoughts?

8. The grey area - I keep thinking that packages/builds need to be released based on a particular set of test criteria (grey), but test results should show all failures/errors/skips. Thoughts? If agreed, how to accomplish? Ruby code and regex fun with test result log files?

9. By the way, ever tried to use RubyCI on a tablet? I tried to do so to see what gcc version *nix builds were using. I'll write code to put results on any web site rather than use RubyCI. FYI, I wrote all the js & html for my doc site, no jquery. But, doing so still takes time...

Packages

10. ci.ri2 - never tried anything like that. I assume pacman works fine with more than one database? If so, great idea, and thanks for the code.

11. How to structure/build - since dependent package updates will be rare, maybe one repo for all of them, then change the appveyor.yml script (just cd to the folder?) based on what package needs to be built? Seems foolish to build all when only one needs to be built. Or, OpenSSL takes quite a while...

12. Right now, I've got gdbm 1.10, libyaml 0.1.7 static, and openssl 1.1.0f. Are those three ok, and is that all we need?

A task I've been avoiding is creating env's for ruby_2_3, ruby_2_4, and trunk, setting it up so I can apply/reverse shared patches easily, rebuild easily, and run subsets of tests on all three, both in a make env and a straight windows env. I'll get started on that...

Thanks again for your work. What would you like me to start with?

[larskanis]

Ruby 2.3

I already added a ruby-2.3 package in the past, but removed it, because the appveyor run took longer and 2.3-tests failed sometimes. Now, that it is separated into rubyinstaller2-packages this is no longer an issue. We could add a post on https://rubyinstaller.org which tells about ruby 2.3 and ruby-snapshot?

ci.ri2 - never tried anything like that. I assume pacman works fine with more than one database? If so, great idea, and thanks for the code.

The code is derived from https://github.com/Alexpux/MINGW-packages but adds packages signing and daily builds. It is based on bash scripts, git and pacman and doesn't use any ruby code. In contrast rubyinstaller2 is a pure packaging project now, with lots of ruby code but no compile or patch tasks.

How to structure/build - since dependent package updates will be rare, maybe one repo for all of them, then change the appveyor.yml script (just cd to the folder?) based on what package needs to be built? Seems foolish to build all when only one needs to be built. Or, OpenSSL takes quite a while...

This is already solved: Only packages changed by the last commit are rebuilt.

Right now, I've got gdbm 1.10, libyaml 0.1.7 static, and openssl 1.1.0f. Are those three ok, and is that all we need?

I understand the reason for gdbm, but why a different version of libyaml?

[MSP-Greg]

ci.ri2 - Cool. I'll have a look at my package code, and see if the three still build with gcc 7.1.0.

libyaml - Well, I built that package in February. So, I checked a recent mswin build and the code building it, no reference to libyaml. Then checked ruby, found ext/psych/yaml/config.h showing 0.1.7.

Also, compare ruby code with libyaml code. It looks like libyaml 0.1.7 is embedded in ruby, they don't use a library/package. We may not even need a libyaml package to build. As to why I built static, I don't recall...

Edit- checked tagged versions - 2.4.1 has 0.1.7, 2.3.4 has 0.1.6, but ruby_2_3 stable has 0.1.7.

2.3 - the last build I did was 'supercharged', as I loaded all new versions of default extension gems before building; it was being used in an embedded application.

We could add a post on https://rubyinstaller.org which tells about ruby 2.3 and ruby-snapshot?

Maybe get 2.3 and 2.4 on Appveyor? The more people testing with it, the better. I suppose the issue of what Appveyor should name 2.3, since they'll then have both RI and RI2 versions...

I'll try a normal 2.3 build in my system; I'll post the test results.

And, from prev, what would you like me to start with?

[MSP-Greg]

I've build many times with trunk and your patch at ruby-lang.org issue 13748, also built v2.3.4, v2.4.1, ruby_2_3, and ruby_2_4 with gcc 7.1.0. All good.

[larskanis]

Thank you @MSP-Greg for all your testing!

[cyclump]

Is the ci.r2 repo needed for normal use or is it just for testing rubyinstaller builds? It is on my desktop install (and a few other of our developers) and we found it when I was trying to update msys2 (we're behind a corporate firewall). I am trying to figure out if I need it or not. If I need to keep it I have to add a path in our proxy for it.

[larskanis]

@cyclump Most of the packages are used from the standard MSYS2 repositories. But for building RubyInstaller2 we need some special versions of certain packages which are delivered through the ci.ri2 repository. This is added automatically when building RubyInstaller2. It is not required when using the binary RubyInstaller distribution. The ci.ri2 repository is filled through these receipts: https://github.com/oneclick/rubyinstaller2-packages