search

newrelic / newrelic-ruby-agent

New Relic RPM Ruby Agent
https://docs.newrelic.com/docs/apm/agents/ruby-agent/getting-started/introduction-new-relic-ruby/
Apache License 2.0
1.2k stars 599 forks source link

CI: Update Jruby multiverse CI jobs to ubuntu 22.04 #1173

Closed tannalynn closed 2 years ago

tannalynn commented 2 years ago

Our jruby multiverse suites are still running on 18.04. All of our other test jobs are running on 20.04, so it would be great to get jruby updated to also run on 20.04. And now that 22.04 is out, we may as well upgrade all the way to that version. Our ci_cron workflow was running it on 20.04 and there are some odd failures that cannot be reproduced locally, or in ubuntu 18.04. All typhoeus suite are failing with 11 failed tests. Most tests are failing with the same errors, so I've only included examples of the 3 different failures.

  2) Failure:
TyphoeusTest#test_crossapp_metrics_allow_valid_utf8_characters [/home/runner/work/newrelic-ruby-agent/newrelic-ruby-agent/test/new_relic/http_client_test_cases.rb:381]:
Expected [:uri] to include :transaction_guid.

  4) Failure:
TyphoeusTest#test_get_with_host_header_lowercase [/home/runner/work/newrelic-ruby-agent/newrelic-ruby-agent/test/new_relic/http_client_test_cases.rb:141]:
Expected # encoding: ASCII-8BIT
"" to match /<head>/i.

  8) Failure:
TyphoeusTest#test_doesnt_affect_the_request_if_an_exception_is_raised_while_finishing_tracing [/home/runner/work/newrelic-ruby-agent/newrelic-ruby-agent/test/new_relic/http_client_test_cases.rb:429]:
--- expected
+++ actual
@@ -1 +1,2 @@
-"<html><head><title>FakeExternalServer status</title></head><body>The FakeExternalServer is rockin'</body></html>"
+# encoding: ASCII-8BIT
+""

One failure from the Rails suite

Failure:
ErrorsWithoutSSCTest#test_captured_errors_should_not_include_custom_params_if_config_says_no [/home/runner/work/newrelic-ruby-agent/newrelic-ruby-agent/test/new_relic/multiverse_helpers.rb:201]:
Expected: 1
  Actual: 2
fallwith commented 2 years ago

We'll need to complete this before 2022-12-01, when GHA drops support for 18.04.

fallwith commented 2 years ago

New PR #1327 is failing for different reasons it appears...

fallwith commented 2 years ago

Going from Ubuntu 18 to 22 produces fewer errors than going to 20, so perhaps we should focus on 22. See #1330

fallwith commented 2 years ago

This issue has been updated to reflect a desire to migrate to Ubuntu 22. There are only 2 suite failures for that version and going there now saves us from having to migrate again when 20 reaches EOL.

fallwith commented 2 years ago

With the jruby:9.3.7.0-jre8 Docker image, all tests pass.

With the following Ubuntu 22.04 Dockerfile, all tests pass:

FROM ubuntu:22.04

ARG DEBIAN_FRONTEND=noninteractive
ENV TZ Etc/UTC 

RUN apt-get update && apt-get install -y ca-certificates libc6-dev git curl autoconf bison build-essential libssl-dev libyaml-dev libreadline6-dev zlib1g-dev libncurses5-dev libffi-dev libgdbm6 libgdbm-dev libdb-dev default-jre tzdata --no-install-recommends && rm -rf /var/lib/apt/lists/*

ENV JRUBY_VERSION 9.3.7.0
ENV JRUBY_SHA256 94a7a8b3beeac2253a8876e73adfac6bececb2b54d2ddfa68f245dc81967d0c1
RUN mkdir /opt/jruby \
  && curl -fSL https://repo1.maven.org/maven2/org/jruby/jruby-dist/${JRUBY_VERSION}/jruby-dist-${JRUBY_VERSION}-bin.tar.gz -o /tmp/jruby.tar.gz \
  && echo "$JRUBY_SHA256 /tmp/jruby.tar.gz" | sha256sum -c - \
  && tar -zx --strip-components=1 -f /tmp/jruby.tar.gz -C /opt/jruby \
  && rm /tmp/jruby.tar.gz \
  && update-alternatives --install /usr/local/bin/ruby ruby /opt/jruby/bin/jruby 1

ENV PATH /opt/jruby/bin:$PATH
ENV SERIALIZE 1

RUN mkdir -p /opt/jruby/etc \
    && { \
        echo 'install: --no-document'; \
        echo 'update: --no-document'; \
    } >> /opt/jruby/etc/gemrc

CMD ["irb"]

So whatever issues we're seeing in CI appear to be related to one of these:

and not one of these:

Perhaps we can special case CRuby 2.2 and JRuby to bypass one or both of the aforementioned suspects.

fallwith commented 2 years ago

Using a GHA Ubuntu 22 image with JRuby installed manually via Maven instead of using setup-ruby produces the same test failures that only happen with GHA.

So we can't reproduce the issue with Ubuntu 22 outside of GitHub, and within GitHub, the use of setup-ruby doesn't appear to be a factor.

Perhaps a GHA Ubuntu 20 or a non-GHA Ubuntu 22 could be used.

fallwith commented 2 years ago

Addressed by #1173