GoogleContainerTools / jib

🏗 Build container images for your Java applications.
Apache License 2.0
13.68k stars 1.44k forks source link

Enable cache configuration in maven mojos #2970

Open rmannibucau opened 3 years ago

rmannibucau commented 3 years ago

Until I miss it there is no way to configure the cache directories in the mojo configuration. My use case is the following, project layout looks like:

+- module1
+- module2
+- .....
+- images
     +- module1-image
     +- module2-image
     +- ...

The containerization is configured in images/pom.xml since it is using the same pattern for all images, then running mvn package jib:dockerBuild on images/ folder makes all images at once. This pattern is very convenient, however each image has its own caching until jib.baseImageCache (and app one) is set.

This issue is about enabling:

  1. To set it in the pom (likely project.parent.build.directory/jib-cache for ^^ layout)
  2. Ensure it is concurrent build friendly (there is a locking on the dir?)
chanseokoh commented 3 years ago

Sorry, I'm a bit confused about how exactly you want to configure the base image cache and the application cache for each sub-module, and for what. I also have trouble relating it to enabling mojo configurations.

however each image has its own caching until jib.baseImageCache ... is set.

This isn't true for the base image cache. The base image cache (e.g., often ~/.cache/google-cloud-tools-java/jib/ on Linux) is a central cache intended to be shared by all Jib-based tools (jib-gradle-plugin, jib-maven-plugin, jib-core, jib-cli, ...) and their projects (and sub-projects). Locking is currently used only for saving small JSON files (manifests and image configurations). Although it's possible multiple Jib-based processes or threaded Maven builds may initially download the same blob concurrently in a race condition (#2007), I don't think there's a performance disadvantage for concurrent builds once blobs are cached.

And are you trying to set separate base image caches for each module?-image? What for?

And for the application layer cache, if you want to have a shared cache, you can set jib.applicationCache?


As for why we don't normally have mojo configuration for these types of configurations, it's because they are highly system dependent. It's arguable, but for example, we don't want to see a lot of git source repos containing pom.xml with <directory>C:\some\base\image\cache\dir</directory> in it.

rmannibucau commented 3 years ago

@chanseokoh what i want to achieve is to ensure all image modules share the same cache but that the cache is per project (to avoid concurrent builds issue or permission issues in some docker setup). So build cache will be something like /images/target/cache or /images/.cache depending the setup. I see it as frontend-maven-plugin which downloads node+npm/yarn/.... Indeed you can cache it globally, it fits well dev machines, but rarely CI (mutualized or shared, both cases have pitfalls). So generally the config is a maven property, defaulting to ~/something in the dev settings.xml and something else in the CI but this is up to the dev/ops doing the setup, not the plugin to decide IMHO. Last point is that -Djib.xxxx is bothering compared to "" (nothing since it has a good default in the pom ;)). In particular because it is not a single folder there so it is quickly verbose. Hope it makes more sense with this more verbose version.

chanseokoh commented 3 years ago

Okay, just to confirm, the issue is simply asking for adding Maven/Gradle plugin configuration counterparts for the currently-system-property-only jib.applicationCache and jib.baseImageCache, right? (Not that Jib has an issue that prevents you from having all image modules share the same cache but that the cache is per project, right?)

rmannibucau commented 3 years ago

@chanseokoh yes, just exposing jib-core config through of the mojo (gradle task if needed).