search

AKSW / RDFUnit

An RDF Unit Testing Suite
http://RDFUnit.aksw.org
Apache License 2.0
155 stars 40 forks source link

Issue running RDFUnit Docker in a Kubernetes environment #98

Closed vemonet closed 4 years ago

vemonet commented 4 years ago

Expected Behavior

Using RDFUnit in a Kubernetes environment

Actual Behavior

When running bin/rdfunit from inside a Docker container running in a Kubernetes pod we are facing the following issue: 

[ERROR] No plugin found for prefix 'exec' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (/root/.m2/
repository), central (https://repo.maven.apache.org/maven2)] -> [Help 1]

Steps to Reproduce the Problem

  1. Install kubectl and Argo on local machine (following this tutorial: https://maastrichtu-ids.github.io/dsri-documentation/docs/guide-local-install#install-kubectl )

  2. Run aksw/rdfunit in a Kubernetes pod It can be done reusing this pod definition: https://github.com/MaastrichtU-IDS/d2s-argo-workflows/blob/cd8b1432940595e6ff52b7efaa339f5d653aa609/tests/test-devnull-pod.yaml

  3. Connect to it to run bin/rdfunit

Specifications

Remarks

I realized there are a few issues in the Dockerfile such as a mkdir line that contains a typo

I think it can be solved by rewriting the Dockerfile to properly generates a packaged jar and using this jar after with java -jar (instead of using mvn -pl)

vemonet commented 4 years ago

Here is an example of a Dockerfile to build Java we use usually (1 stage with maven to build the jar, 1 stage with a light JRE to execute the jar : https://github.com/MaastrichtU-IDS/xml2rdf/blob/master/Dockerfile

Compiling a jar allow you to avoid any issues related to changing the path where the application is used within your container

Commands to run the test pod and connect to it (from the d2s-argo-workflow repo):

kubectl create -f tests/test-devnull-pod.yaml
kubectl exec -it test-devnull-pod -- /bin/bash
vemonet commented 4 years ago

I fixed RDFUnit to be packaged as a standalone jar in the RDFUnit Docker container. So using it from any path with any workdir set will work (available at https://hub.docker.com/repository/docker/umids/rdfunit )

What has been done:

See commits:

See the new Dockerfile It is working perfectly fine, let me know if that interest you

We could avoid to change the pom.xml by selection the cli-standalone profile when running mvn package, but not sure how to do it

jimkont commented 4 years ago

Thanks for the report @vemonet I agree that using a fat jar is a better option for running RDFUnit in docker

I like your change, my only concern is that by adding the shade plugin inside the default maven profile, it will be running for every maven execution and possibly delay other dev processes.

We could easily add it in a separate profile (i.e. standaloneJar) and add -PstandaloneJar in the docker command, wdyt?

vemonet commented 4 years ago

Hi @jimkont , yes that was my thought also

I rewrote the change to edit only the Dockerfile to properly use the cli-standalone profile when running mvn package

See commit https://github.com/vemonet/RDFUnit/commit/2452b1b0c019524ba35e0e4f4368abd1776521d3

And see the Dockerfile: https://github.com/vemonet/RDFUnit/blob/develop/Dockerfile

I kept /usr/src/myapp as workdir, but we could use something shorter if you want, like /app

Additionally this also fix RDFUnit to be used in CWL workflows (CWL redefine the Docker workdir) :slightly_smiling_face:

I did a pull request if that works for you: https://github.com/AKSW/RDFUnit/pull/99

k00ni commented 4 years ago

What is the status here?

jimkont commented 4 years ago

@vemonet can you check if we can close this issue? Is there anything else pending after your Merge request (https://github.com/AKSW/RDFUnit/pull/99)?

vemonet commented 4 years ago

@jimkont it is working as expected now, thanks a lot!