search

sparklemotion / nokogiri

Nokogiri (鋸) makes it easy and painless to work with XML and HTML from Ruby.
https://nokogiri.org/
MIT License
6.13k stars 897 forks source link

investigate why CI started failing on the Windows Ruby 2.5 builds #2354

Closed flavorjones closed 2 years ago

flavorjones commented 2 years ago

Same commit was fine one day and reliably segfaults the next:

Good:

Bad:

All these runs were for the same commit, 3800769e37719beb09dc85718527105d1676dc17

Looking for differences in the logs:

virtual environment:

--- good.actions.log2   2021-11-04 13:14:27.862287741 -0400
+++ bad.actions.log2    2021-11-04 13:14:23.772273959 -0400
@@ -1,7 +1,7 @@
 Can't find any online and idle self-hosted or hosted runner in the current repository, account/organization  that mat
ches the required labels: 'windows-latest'
 Waiting for a self-hosted or a hosted runner to pickup this job...
 Job is waiting for a hosted runner to come online.
-Job is about to start running on the hosted runner: GitHub Actions 12 (hosted)
+Job is about to start running on the hosted runner: GitHub Actions 5 (hosted)
 Current runner version: '2.284.0'
 ##[group]Operating System
 Microsoft Windows Server 2019
@@ -10,9 +10,9 @@
 ##[endgroup]
 ##[group]Virtual Environment
 Environment: windows-2019
-Version: 20211018.0
-Included Software: https://github.com/actions/virtual-environments/blob/win19/20211018.0/images/win/Windows2019-Readm
e.md
-Image Release: https://github.com/actions/virtual-environments/releases/tag/win19%2F20211018.0
+Version: 20211102.4
+Included Software: https://github.com/actions/virtual-environments/blob/win19/20211102.4/images/win/Windows2019-Readm
e.md
+Image Release: https://github.com/actions/virtual-environments/releases/tag/win19%2F20211102.4
 ##[endgroup]
 ##[group]Virtual Environment Provisioner
 1.0.0.0-master-20211022-1
flavorjones commented 2 years ago

The diff between those images' readmes:

--- 20211018.readme 2021-11-04 13:21:39.192621214 -0400
+++ 20211102.readme 2021-11-04 13:22:12.176715972 -0400
@@ -1,10 +1,14 @@
 | Announcements |
 |-|
+| [[all OSs] Selenium server version will be upgraded from 3 to 4 on November, 15](https://github.com/actions/virtual-environments/issues/4376) |
+| [[All OSs] Node.js version 10 will be removed from the images on November, 15](https://github.com/actions/virtual-environments/issues/4356) |
+| [Windows-2016 environment will be removed on March 15, 2022](https://github.com/actions/virtual-environments/issues/4312) |
+| [[All OSs] Go versions less than 1.15 will be removed on November, 8](https://github.com/actions/virtual-environments/issues/4311) |
 | [(Public Beta) Windows Server 2022 with Visual Studio 2022 is now available](https://github.com/actions/virtual-environments/issues/3949) |
 ***
 # Microsoft Windows Server 2019 Datacenter
 - OS Version: 10.0.17763 Build 2237
-- Image Version: 20211018.0
+- Image Version: 20211102.4

 ## Enabled windows optional features
 - Windows Subsystem for Linux [WSLv1]
@@ -23,16 +27,16 @@
 - Ruby 2.5.9p229

 ### Package Management
-- Chocolatey 0.11.2
-- Composer 2.1.9
+- Chocolatey 0.11.3
+- Composer 2.1.11
 - Helm 3.7.0
 - Miniconda 4.10.3 (pre-installed on the image but not added to PATH)
 - NPM 6.14.15
 - NuGet 5.11.0.10
-- pip 21.3 (python 3.7)
+- pip 21.3.1 (python 3.7)
 - Pipx 0.16.4
 - RubyGems 2.7.6.3
-- Vcpkg  (build from master \<699c877>)
+- Vcpkg  (build from master \<85b3115>)
 - Yarn 1.22.17

 #### Environment variables
@@ -42,7 +46,7 @@
 | CONDA                   | C:\Miniconda |

 ### Project Management
-- Ant 1.10.11
+- Ant 1.10.12
 - Gradle 7.2
 - Maven 3.8.3
 - sbt 1.5.5
@@ -50,30 +54,30 @@
 ### Tools
 - 7zip 19.00
 - aria2 1.36.0
-- azcopy 10.12.2
+- azcopy 10.13.0
 - Bazel 4.2.1
 - Bazelisk 1.10.1
 - Bicep 0.4.1008
 - Cabal 3.6.2.0
-- CMake 3.21.3
-- CodeQL Action Bundle 2.6.3-a
+- CMake 3.21.4
+- CodeQL Action Bundle 2.7.0
 - Docker 20.10.7
 - Docker-compose 1.29.2
 - ghc 9.0.1
 - Git 2.33.1
 - Git LFS 3.0.1
-- Google Cloud SDK 360.0.0
+- Google Cloud SDK 363.0.0
 - InnoSetup 6.2.0
 - jq 1.6
 - Kind 0.11.1
-- Kubectl 1.22.2
+- Kubectl 1.22.3
 - Mercurial 5.0
 - Mingw-w64 8.1.0
 - Newman 5.3.0
 - NSIS v3.06.1
 - OpenSSL 1.1.1
 - Packer 1.7.6
-- Pulumi v3.15.0
+- Pulumi v3.16.0
 - R 4.1.1
 - Stack 2.7.3
 - Subversion (SVN) 1.14.1
@@ -84,20 +88,20 @@
 - zstd 1.5.0

 ### CLI Tools
-- Alibaba Cloud CLI 3.0.94
-- AWS CLI 2.2.46
-- AWS SAM CLI 1.33.0
-- AWS Session Manager CLI 1.2.245.0
-- Azure CLI 2.29.0
-- Azure DevOps CLI extension 0.20.0
+- Alibaba Cloud CLI 3.0.97
+- AWS CLI 2.3.0
+- AWS SAM CLI 1.34.1
+- AWS Session Manager CLI 1.2.279.0
+- Azure CLI 2.30.0
+- Azure DevOps CLI extension 0.21.0
 - Cloud Foundry CLI 6.53.0
-- GitHub CLI 2.1.0
+- GitHub CLI 2.2.0
 - Hub CLI 2.14.2

 ### Rust Tools
-- Cargo 1.55.0
-- Rust 1.55.0
-- Rustdoc 1.55.0
+- Cargo 1.56.0
+- Rust 1.56.1
+- Rustdoc 1.56.1
 - Rustup 1.24.3

 #### Packages
@@ -105,32 +109,34 @@
 - cargo-audit 0.15.2
 - cargo-outdated v0.9.17
 - cbindgen 0.20.0
-- Clippy 0.1.55
+- Clippy 0.1.56
 - Rustfmt 1.4.37

 ### Browsers and webdrivers
-- Google Chrome 94.0.4606.81
-- Chrome Driver 94.0.4606.61
-- Microsoft Edge 94.0.992.50
-- Microsoft Edge Driver 94.0.992.50
-- Mozilla Firefox 93.0
+- Google Chrome 95.0.4638.69
+- Chrome Driver 95.0.4638.54
+- Microsoft Edge 95.0.1020.40
+- Microsoft Edge Driver 95.0.1020.40
+- Mozilla Firefox 94.0
 - Gecko Driver 0.30.0
 - IE Driver 3.150.1.1
+- Selenium server 3.141.59

 #### Environment variables
 | Name            | Value                              |
-| --------------- | ---------------------------------- |
+| ----------------- | ------------------------------------------ |
 | CHROMEWEBDRIVER | C:\SeleniumWebDrivers\ChromeDriver |
 | EDGEWEBDRIVER   | C:\SeleniumWebDrivers\EdgeDriver   |
 | GECKOWEBDRIVER  | C:\SeleniumWebDrivers\GeckoDriver  |
+| SELENIUM_JAR_PATH | C:\selenium\selenium-server-standalone.jar |

 ### Java
 | Version             | Vendor          | Environment Variable |
 | ------------------- | --------------- | -------------------- |
-| 8.0.302+8 (default) | Eclipse Temurin | JAVA_HOME_8_X64      |
-| 11.0.12+7           | Eclipse Temurin | JAVA_HOME_11_X64     |
+| 8.0.312+7 (default) | Eclipse Temurin | JAVA_HOME_8_X64      |
+| 11.0.13+8           | Eclipse Temurin | JAVA_HOME_11_X64     |
 | 13.0.2+8.1          | Adopt OpenJDK   | JAVA_HOME_13_X64     |
-| 17.0.0+35           | Eclipse Temurin | JAVA_HOME_17_X64     |
+| 17.0.1+12           | Eclipse Temurin | JAVA_HOME_17_X64     |

 ### Shells
 | Name          | Target                            |
@@ -166,6 +172,7 @@
 | 10.24.1 | x64          |
 | 12.22.7 | x64          |
 | 14.18.1 | x64          |
+| 16.13.0 | x64          |

 #### Python
@@ -195,7 +202,8 @@
 | -------------- | ------------ |
 | 2.7.18         | PyPy 7.3.6 with MSC v.1929 64 bit (AMD64) |
 | 3.6.12         | PyPy 7.3.3 with MSC v.1927 32 bit |
-| 3.7.12         | PyPy 7.3.6 with MSC v.1929 64 bit (AMD64) |
+| 3.7.12         | PyPy 7.3.7 with MSC v.1929 64 bit (AMD64) |
+| 3.8.12         | PyPy 7.3.7 with MSC v.1929 64 bit (AMD64) |

@@ -221,7 +229,7 @@

 ### Database tools
-- Azure CosmosDb Emulator 2.14.3.0
+- Azure CosmosDb Emulator 2.14.4.0
 - DacFx 15.0.5282.3
 - MySQL 5.7.35.0
 - SQLPS 1.0
@@ -520,7 +528,7 @@
 #### Azure Powershell Modules
 | Module  | Version                                                                                                                                                           | Path                           |
 | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------ |
-| Az      | 1.0.0.zip<br>1.6.0.zip<br>2.3.2.zip<br>2.6.0.zip<br>3.1.0.zip<br>3.5.0.zip<br>3.8.0.zip<br>4.3.0.zip<br>4.4.0.zip<br>4.7.0.zip<br>5.5.0.zip<br>5.9.0.zip<br>6.4.0 | C:\Modules\az_\<version\>      |
+| Az      | 1.0.0.zip<br>1.6.0.zip<br>2.3.2.zip<br>2.6.0.zip<br>3.1.0.zip<br>3.5.0.zip<br>3.8.0.zip<br>4.3.0.zip<br>4.4.0.zip<br>4.7.0.zip<br>5.5.0.zip<br>5.9.0.zip<br>6.5.0 | C:\Modules\az_\<version\>      |
 | Azure   | 2.1.0 [Installed]<br>3.8.0.zip<br>4.2.1.zip<br>5.1.1.zip<br>5.3.0                                                                                                 | C:\Modules\azure_\<version\>   |
 | AzureRM | 2.1.0 [Installed]<br>3.8.0.zip<br>4.2.1.zip<br>5.1.1.zip<br>6.7.0.zip<br>6.13.1                                                                                   | C:\Modules\azurerm_\<version\> |

@@ -533,7 +541,7 @@
 | ------------------ | ---------------- |
 | DockerMsftProvider | 1.0.0.8          |
 | MarkdownPS         | 1.9              |
-| Microsoft.Graph    | 1.7.0            |
+| Microsoft.Graph    | 1.8.0            |
 | Pester             | 3.4.0<br>5.3.1   |
 | PowerShellGet      | 1.0.0.1<br>2.2.5 |
 | PSScriptAnalyzer   | 1.20.0           |
@@ -554,7 +562,7 @@
 | Google APIs                | addon-google_apis-google-21<br>addon-google_apis-google-22<br>addon-google_apis-google-23<br>addon-google_apis-google-24                                                                                                                                                                        |
 | Google Play services       | 49                                                                                                                                                                                                                                                                                              |
 | Google Repository          | 58                                                                                                                                                                                                                                                                                              |
-| NDK                        | 21.4.7075529 (default)<br>22.1.7171670<br>23.0.7599858                                                                                                                                                                                                                                          |
+| NDK                        | 21.4.7075529 (default)<br>22.1.7171670<br>23.1.7779620                                                                                                                                                                                                                                          |
 | SDK Patch Applier v4       | 1                                                                                                                                                                                                                                                                                               |

 #### Environment variables
@@ -562,7 +570,7 @@
 | ----------------------- | ---------------------------------------------------------------------------- |
 | ANDROID_HOME            | C:\Android\android-sdk                                                       |
 | ANDROID_NDK_HOME        | C:\Android\android-sdk\ndk-bundle -> C:\Android\android-sdk\ndk\21.4.7075529 |
-| ANDROID_NDK_LATEST_HOME | C:\Android\android-sdk\ndk\23.0.7599858                                      |
+| ANDROID_NDK_LATEST_HOME | C:\Android\android-sdk\ndk\23.1.7779620                                      |
 | ANDROID_NDK_PATH        | C:\Android\android-sdk\ndk-bundle -> C:\Android\android-sdk\ndk\21.4.7075529 |
 | ANDROID_NDK_ROOT        | C:\Android\android-sdk\ndk-bundle -> C:\Android\android-sdk\ndk\21.4.7075529 |
 | ANDROID_SDK_ROOT        | C:\Android\android-sdk                                                       |
flavorjones commented 2 years ago

mini_portile2 seems fine on this version of windows runner: https://github.com/flavorjones/mini_portile/runs/4108367507?check_suite_focus=true

MSP-Greg commented 2 years ago

I've repo'd locally. Both asserts generate a SIGSEGV.

It's not easy to determine, but I believe the MSYS2 MinGW gcc was updated to 11.2.0 somewhere around 04-Nov. Ruby 2.5.9 was built with 10.2.0.

https://github.com/sparklemotion/nokogiri/blob/cc5a7ea9cca6c6efd6175bd35f81382517942e1b/test/xml/test_reader.rb#L90-L97

flavorjones commented 2 years ago

@MSP-Greg Thank you for taking a look at this! To make sure I understand what you're saying: Ruby was compiled with 10.2.0 but the extension is being compiled with 11.2.0?

The argument-checking code for both of those functions is pretty straightforward and it's not obvious to me why these (and only these) code paths would segfault ...

static VALUE
from_io(int argc, VALUE *argv, VALUE klass)
{
  VALUE rb_io, rb_url, encoding, rb_options;
  xmlTextReaderPtr reader;
  const char *c_url      = NULL;
  const char *c_encoding = NULL;
  int c_options           = 0;
  VALUE rb_reader, args[3];

  rb_scan_args(argc, argv, "13", &rb_io, &rb_url, &encoding, &rb_options);

  if (!RTEST(rb_io)) { rb_raise(rb_eArgError, "io cannot be nil"); }
static VALUE
from_memory(int argc, VALUE *argv, VALUE klass)
{
  VALUE rb_buffer, rb_url, encoding, rb_options;
  xmlTextReaderPtr reader;
  const char *c_url      = NULL;
  const char *c_encoding = NULL;
  int c_options           = 0;
  VALUE rb_reader, args[3];

  rb_scan_args(argc, argv, "13", &rb_buffer, &rb_url, &encoding, &rb_options);

  if (!RTEST(rb_buffer)) { rb_raise(rb_eArgError, "string cannot be nil"); }

If you have any insights or advice, I'm all ears. Otherwise I may try to just skip these tests for Ruby 2.5 on Windows.

MSP-Greg commented 2 years ago

To make sure I understand what you're saying: Ruby was compiled with 10.2.0 but the extension is being compiled with 11.2.0?

Yes. The Windows build tools (MSYS2 MinGW 32 & 64 bit) are updated every time there's a new version release of the Actions Windows images. On windows-2019, the 32 & 64 bit gcc tools were included with the image. With windows-2022, the gcc tools were removed, and some 'generic' build tools (worked with both 32 & 64 bit) were also removed.

If you have any insights or advice

I haven't seen this problem elsewhere, so I'm confused. I've seen issues where someone tries to build against something like 2.4.0 or 2.5.0, but not with a 'recent' build. I was going to look at it a bit more tonite...

flavorjones commented 2 years ago

@MSP-Greg That test is skipped on Windows in https://github.com/sparklemotion/nokogiri/pull/2384 and, sure enough, the rest of the suite is green. Very strange, but I'm going to try not to think about it too hard :zany_face:

flavorjones commented 2 years ago

Closing, because #2396 dropped 2.5 out of the testing matrix. Thanks again for your help, @MSP-Greg!