Failures of docker to build (force downgrade of Ruby to 3.0 due to Nokogiri 1.13 entity issue with Relaton)

[opoudjis]

https://github.com/metanorma/metanorma-docker/runs/6362390329?check_suite_focus=true

[ronaldtse]

I'm thinking that we no longer need the mn-* versions of the containers, especially since they are not properly tested and that the release flow now is much more stable than before.

Thoughts @CAMOBAP ?

[CAMOBAP]

Yep, they are not properly tested but as far as I understand they may be useful during testing some in features in-development (between releases). But personally, I never used them

@ronaldtse @opoudjis (or maybe someone else in the team) have you used mn-* in some cases?

**investigating

[ronaldtse]

@CAMOBAP we only used them as hotfixes for immediate problems that cannot wait for 2 weeks until the next release cycle -- we do not currently have a "hotfix" branching practice, @opoudjis is not a fan of it.

The problem now is that the mn-* are failing.

[CAMOBAP]

Fixed push-tag workflow (to ignore branch) mn containers removed from CI

Once passed I will retag

[CAMOBAP]

@opoudjis @ronaldtse can you confirm that all required gems are released?

Fatal Error: wrong number of arguments (given 3, expected 2)
/usr/local/bundle/gems/isodoc-i18n-1.0.5/lib/isodoc/i18n.rb:57:in `initialize': wrong number of arguments (given 3, expected 2) (ArgumentError)
    from /usr/local/bundle/gems/metanorma-iso-2.0.8.1/lib/isodoc/iso/init.rb:19:in `new'
    from /usr/local/bundle/gems/metanorma-iso-2.0.8.1/lib/isodoc/iso/init.rb:19:in `i18n_init'
    from /usr/local/bundle/gems/metanorma-standoc-2.0.7/lib/metanorma/standoc/utils.rb:64:in `isodoc'
    from /usr/local/bundle/gems/metanorma-standoc-2.0.7/lib/metanorma/standoc/base.rb:90:in `init_i18n'
    from /usr/local/bundle/gems/metanorma-standoc-2.0.7/lib/metanorma/standoc/base.rb:34:in `init'
    from /usr/local/bundle/gems/metanorma-iso-2.0.8.1/lib/metanorma/iso/base.rb:45:in `init'
    from /usr/local/bundle/gems/metanorma-standoc-2.0.7/lib/metanorma/standoc/base.rb:1[13](https://github.com/metanorma/metanorma-docker/runs/6366379318?check_suite_focus=true#step:8:13):in `document1'
    from /usr/local/bundle/gems/metanorma-standoc-2.0.7/lib/metanorma/standoc/base.rb:103:in `document'
    from /usr/local/bundle/gems/metanorma-standoc-2.0.7/lib/metanorma/standoc/utils.rb:13:in `convert'
    from /usr/local/bundle/gems/asciidoctor-2.0.17/lib/asciidoctor/document.rb:955:in `convert'
    from /usr/local/bundle/gems/asciidoctor-2.0.17/lib/asciidoctor/convert.rb:120:in `convert'
    from /usr/local/bundle/gems/metanorma-1.4.7.1/lib/metanorma/input/asciidoc.rb:18:in `process'
    from /usr/local/bundle/gems/metanorma-1.4.7.1/lib/metanorma/processor.rb:21:in `input_to_isodoc'
    from /usr/local/bundle/gems/metanorma-1.4.7.1/lib/metanorma/compile.rb:59:in `process_input_adoc'
    from /usr/local/bundle/gems/metanorma-1.4.7.1/lib/metanorma/compile.rb:42:in `process_input'
    from /usr/local/bundle/gems/metanorma-1.4.7.1/lib/metanorma/compile.rb:33:in `compile'
    from /usr/local/bundle/gems/metanorma-cli-1.5.[15](https://github.com/metanorma/metanorma-docker/runs/6366379318?check_suite_focus=true#step:8:15)pre1/lib/metanorma/cli/compiler.rb:45:in `compile_file'
    from /usr/local/bundle/gems/metanorma-cli-1.5.15pre1/lib/metanorma/cli/compiler.rb:30:in `compile'
    from /usr/local/bundle/gems/metanorma-cli-1.5.15pre1/lib/metanorma/cli/compiler.rb:35:in `compile'
    from /usr/local/bundle/gems/metanorma-cli-1.5.15pre1/lib/metanorma/cli/site_generator.rb:89:in `compile'
    from /usr/local/bundle/gems/metanorma-cli-1.5.15pre1/lib/metanorma/cli/site_generator.rb:29:in `block in generate'
    from /usr/local/bundle/gems/metanorma-cli-1.5.15pre1/lib/metanorma/cli/site_generator.rb:29:in `each'
    from /usr/local/bundle/gems/metanorma-cli-1.5.15pre1/lib/metanorma/cli/site_generator.rb:29:in `generate'
    from /usr/local/bundle/gems/metanorma-cli-1.5.15pre1/lib/metanorma/cli/site_generator.rb:24:in `generate'
    from /usr/local/bundle/gems/metanorma-cli-1.5.15pre1/lib/metanorma/cli/commands/site.rb:25:in `generate'
    from /usr/local/bundle/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'
    from /usr/local/bundle/gems/thor-hollaback-0.2.1/lib/thor/hollaback.rb:68:in `run'
    from /usr/local/bundle/gems/thor-1.0.1/lib/thor/invocation.rb:127:in `invoke_command'
    from /usr/local/bundle/gems/thor-1.0.1/lib/thor.rb:392:in `dispatch'
    from /usr/local/bundle/gems/thor-1.0.1/lib/thor/invocation.rb:1[16](https://github.com/metanorma/metanorma-docker/runs/6366379318?check_suite_focus=true#step:8:16):in `invoke'
    from /usr/local/bundle/gems/thor-1.0.1/lib/thor.rb:[24](https://github.com/metanorma/metanorma-docker/runs/6366379318?check_suite_focus=true#step:8:24)3:in `block in subcommand'
    from /usr/local/bundle/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'
    from /usr/local/bundle/gems/thor-hollaback-0.2.1/lib/thor/hollaback.rb:68:in `run'
    from /usr/local/bundle/gems/thor-1.0.1/lib/thor/invocation.rb:127:in `invoke_command'
    from /usr/local/bundle/gems/thor-1.0.1/lib/thor.rb:392:in `dispatch'
    from /usr/local/bundle/gems/thor-1.0.1/lib/thor/base.rb:485:in `start'
    from /usr/local/bundle/gems/metanorma-cli-1.5.15pre1/lib/metanorma/cli.rb:34:in `start'
    from /usr/local/bundle/gems/metanorma-cli-1.5.15pre1/exe/metanorma:[25](https://github.com/metanorma/metanorma-docker/runs/6366379318?check_suite_focus=true#step:8:25):in `block in <top (required)>'
    from /usr/local/bundle/gems/metanorma-cli-1.5.15pre1/exe/metanorma:[43](https://github.com/metanorma/metanorma-docker/runs/6366379318?check_suite_focus=true#step:8:43):in `<top (required)>'
    from /usr/local/bundle/bin/metanorma:25:in `load'
    from /usr/local/bundle/bin/metanorma:25:in `<main>'

[CAMOBAP]

Quick update.

For some reason metanorma-docker wasn't get repository_dispatch event from metanorma-cli CI about 1.5.15.pre.alpha release

In CI logs I clearly see that metanorma-cli did this notification https://github.com/metanorma/metanorma-cli/runs/6362206424?check_suite_focus=true#step:4:21

I have updated the version and triggered the release manually https://github.com/metanorma/metanorma-docker/actions/runs/2301597374

I will watch for this run just in case

[CAMOBAP]

Nogokiri prebuilt gems have a restriction: < 3.1.DEV https://rubygems.org/gems/nokogiri/versions/1.12.5-x86-linux

So it looks like we need to migrate to nokogiri 1.13. It have >= 2.6, < 3.2.DEV which looks acceptable for us

[CAMOBAP]

@opoudjis @ronaldtse @andrew2net is that ok to upgrade nokogiri version to 1.13?

[andrew2net]

@CAMOBAP relaton has an issue when renders XML. Nokogiri v 1.13 instead of escaping entities like & renders it as space.

    it "escape &" do
      xml = Nokogiri::XML::Builder.new do |b|
        b.localized_string { subject.to_xml(b) }
      end
      expect(xml.doc.root.to_s).to be_equivalent_to <<~XML
        <localized_string language="en" script="Latn">
          content &amp; character to escape
        </localized_string>
      XML
    end

[CAMOBAP]

@andrew2net it sounds like an issue in nokogiri, is there some ticket reported for nokogiri?

@ronaldtse is it ok for us to downgrade ruby version in container to 3.0.x

P.S. I checking possibility to avoid rebuild gems for docker as an workaround

[ronaldtse]

@ronaldtse is it ok for us to downgrade ruby version in container to 3.0.x

Definitely.

[ronaldtse]

@CAMOBAP relaton has an issue when renders XML. Nokogiri v 1.13 instead of escaping entities like & renders it as space.

    it "escape &" do
      xml = Nokogiri::XML::Builder.new do |b|
        b.localized_string { subject.to_xml(b) }
      end
      expect(xml.doc.root.to_s).to be_equivalent_to <<~XML
        <localized_string language="en" script="Latn">
          content &amp; character to escape
        </localized_string>
      XML
    end

@andrew2net Is this something Relaton can fix?

[andrew2net]

@andrew2net Is this something Relaton can fix?

@ronaldtse I've tried but didn't find a solution

[ronaldtse]

@CAMOBAP can we just downgrade Ruby to 3.0? Without a successful Docker integration test we can't release metanorma-cli, which means that the gem stack is broken (I can't compile a document too).

[CAMOBAP]

@CAMOBAP can we just downgrade Ruby to 3.0?

Yep it's done locally

Without a successful Docker integration test we can't release metanorma-cli, which means that the gem stack is broken (I can't compile a document too).

Docker will fail with

Bundler could not find compatible versions for gem "metanorma-nist":
  In Gemfile:
    metanorma-cli (= 1.5.15.pre.alpha) was resolved to 1.5.15.pre.alpha, which depends on
      metanorma-nist (~> 2.1.0)

Could not find gem 'metanorma-nist (~> 2.1.0)', which is required by gem 'metanorma-cli (= 1.5.15.pre.alpha)', in rubygems repository https://rubygems.org/ or installed locally.

Should we add our private Github Packages, just for test 1.5.15.pre.alpha ?

[ronaldtse]

@CAMOBAP we should not have metanorma-nist in metanorma-cli, that was removed in https://github.com/metanorma/metanorma-cli/issues/277

[CAMOBAP]

Done