Following up on the discussion with @aams-telefonica about compiling Flink applications, we must automate the generation of JARs at the time when Docker image is built.
In Docker this is possible by using a multi-stage image, where the first stage compiles the code with maven, and the second collects the JAR from the first stage. An example of a multi-stage image that compiles Java code with maven is described here.
PROBLEM DESCRIPTION
The actual way of deploying a flink application is to use Maven locally to package the application into a .jar, that jar is uploaded to Github and used into a Dockerfile to be copied with the flink cluster. This method makes difficult to maintain changes into the applications because every modification will later need to delete the previous .jar file and upload a new one.
The solution is to copy the source code into the image and not the .jar file, later use the installed dependencies of Maven inside of the image to package the application. In order to reduce significantly the size of the builds, Docker multi-stage builds can be used so the final image only contains the generated .jar and not the Maven libraries and the source code.
DESCRIPTION
Following up on the discussion with @aams-telefonica about compiling Flink applications, we must automate the generation of JARs at the time when Docker image is built.
In Docker this is possible by using a multi-stage image, where the first stage compiles the code with maven, and the second collects the JAR from the first stage. An example of a multi-stage image that compiles Java code with maven is described here.
PROBLEM DESCRIPTION
The actual way of deploying a flink application is to use Maven locally to package the application into a .jar, that jar is uploaded to Github and used into a Dockerfile to be copied with the flink cluster. This method makes difficult to maintain changes into the applications because every modification will later need to delete the previous .jar file and upload a new one.
The solution is to copy the source code into the image and not the .jar file, later use the installed dependencies of Maven inside of the image to package the application. In order to reduce significantly the size of the builds, Docker multi-stage builds can be used so the final image only contains the generated .jar and not the Maven libraries and the source code.
A possibly useful example can be read in how to dockerize maven project.
DOCUMENTATION