Config.yml validator code integration in shared library

[samuelbernardolip]

Schema validator for config.yml is developed in groovy language using gradle builder.

To import the code into a Jenkins shared library there are three possible solutions:

• Using folder based shared library it requires to convert the code into Java so it is possible to use a plugin to deliver a new step definition with the required jars.
• Using global shared library it allows to use @Grab annotation directly since the code is evaluated before sandbox runtime. @Grab can be used only with trusted libraries.
• Run groovy script in a JVM launched from sh step. This approach will allow to use folder based shared library that provides a better portability, but code will run independent from the library as a stage.

The adopted solution for the time being was the global shared library, that allow us to use current groovy code.

[samuelbernardolip]

There is another option that consists on the following:

Since Groovy and Shared Groovy Libraries plugin are the ones that define the build loader, my hint is to start building another plugin that could inject the required dependencies. When developing a plugin we can import code from other plugins, we can tackle the Grab requirement distributing a plugin for that. That IndigoGrab loader plugin will have a dependency with Groovy and Shared Groovy Libraries plugins and it will have all dependencies hard coded without requiring any action from the user side (just like a global shared library only to import the required packages for the project).

[samuelbernardolip]

Solution being tested for the time being, surpassing the @Grab limitation (jars not being loaded in slave side), will call the system class loader to import the required jars from library resources folder. https://tcollignon.github.io/2017/07/10/How-To-Use-third-party-libraries-in-Jenkins-Pipeline.html

There is also the following articles where is explained why using @Grab brings security issues to the Jenkins sandbox: https://blog.alertlogic.com/deep-dive-on-jenkins-pipeline-plugin-remote-code-execution/ https://support.cloudbees.com/hc/en-us/articles/360017530472-Pipeline-Script-Security#howaresharedlibrariesaffectedbyscriptsecurity

We should avoid using it inside the library even when used as global shared library.

[samuelbernardolip]

Using Jenkins system classloader didn't work because it don't provide addURL and code is compiled before the jars are loaded. The alternative solution was to run the script inline using GroovyShell in runtime. This was also not possible because Jenkins security blocks the use of CompilerConfiguration required by GroovyShell:

org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use new org.codehaus.groovy.control.CompilerConfiguration

All work is in sl-classloader branch.

Referencies: https://tcollignon.github.io/2017/07/10/How-To-Use-third-party-libraries-in-Jenkins-Pipeline.html https://issues.jenkins-ci.org/browse/JENKINS-43844 https://www.codota.com/code/java/classes/groovy.lang.GroovyShell http://docs.groovy-lang.org/latest/html/documentation/guide-integrating.html https://docs.groovy-lang.org/latest/html/api/groovy/lang/Script.html https://www.programcreek.com/java-api-examples/?class=groovy.lang.GroovyObject&method=invokeMethod

[samuelbernardolip]

Current state of the three approaches are the following:

1. Using a plugin to integrate validator code is not working yet because there is a problem with data binding after translating the Groovy code into Java[1][2][3]
2. Global library usage to import the dependencies is not sufficient because Jenkins Security requirements limit many calls in sandbox environment [4][5]
3. Run validator as an independent application in a step is the current working solution [6][7][8]

[1] https://github.com/WORSICA/indigo-dependencies [2] https://jenkins.eosc-synergy.eu/job/test/job/indigo-dependencies-plugin/ [3] https://github.com/WORSICA/jenkins-pipeline-library/tree/indigo-dependencies [4] https://github.com/WORSICA/jenkins-pipeline-library/tree/sl-classloader [5] https://jenkins.eosc-synergy.eu/job/test/job/shared-library-classloader/ [6] https://github.com/WORSICA/jenkins-pipeline-library/tree/docker-compose [7] https://github.com/EOSC-synergy/jpl-validator [8] https://jenkins.eosc-synergy.eu/job/test/job/test-library-v2/

[orviz]

Once the plugin is available --option 1)-- it will be integrated together with the jpl-validator --option 3)--, so that if the plugin is not found in runtime, the jpl-validator will be used.

[samuelbernardolip]

Since we have already tested all options and concluded that only option 1 and 3 are possible to implement due to Jenkins sandbox restrictions, this issue can be closed since we have the solution. Option 1: already in release 2.0.0 Option 3: to be available as issue #35 is fixed