GoogleContainerTools / jib

🏗 Build container images for your Java applications.
Apache License 2.0
13.68k stars 1.44k forks source link

Make used Gradle SourceSet and Configuration configurable #1778

Closed remmeier closed 3 years ago

remmeier commented 5 years ago

The Jib Gradle plugin is hard-coded to make use of the "main" classpath and "runtime" configuration. It should be possible to specify those two values in the JibExtension to allow customizations:

Many/most other Gradle plugins allow to do the same thing to maintain flexiblity of what gets touched by plugin. And effort for allowing that is minimal.

The ticket is related to https://github.com/GoogleContainerTools/jib/issues/894, but as requested created a ticket on its own. In most cases the changes requested here should be simpler and suffice for developers to allow doing customizations. The other ticket then would allow even more customization at expensve of additional complexity.

loosebazooka commented 5 years ago

add further libraries to the classpath for Docker only

Can you explain this point a little more? It seems like you can either use the runtime scope or the extraDirectories option.

remove libraries from the classpath as they are already available in a base image.

This can be handled by the compileOnly or provided mechanisms?

change the source set because the projects main "purpose" is not being a docker image, but something else and have docker-related sources of the project in a non-main source set. We have that pattern quite frequently when we build plugins for other system that are then installed as initContainer. So the main artifact of the project would be the "plugin", whereas the initContainer is more a side-thing related to deployment. Currently we are forced to make setup two projects rather than just having two source sets.

I'm a little confused about this point. In reality it sounds like two projects IS what you want. They are separate artifacts and that are somewhat related.

remmeier commented 5 years ago

I'm a little confused about this point. In reality it sounds like two projects IS what you want. They are separate artifacts and that are somewhat related.

in this case the created images/initContainer are more of a build artifact rather than something warranting a project on its own. Examples where we have this for example are for Jenkins, Elastic and R3 Corda. We have around a dozen such projects and more coming. So what we would like to be possible to do is a Gradle plugin that takes that plugin project and jib and transparently setups an image/initContainer as part of the build. Important here are two things: the jib-related things cannot be on the runtime classpath and all logic to create the image can be put in a reusable fashion into a Gradle plugin.

add further libraries to the classpath for Docker only Can you explain this point a little more? It seems like you can either use the runtime scope or the extraDirectories option.

e.g. if you want to have a project both as RPM and Image. For either we end up with slightly different runtime classpaths. Goes a bit into a similar direction as above where one could have one rather than three projects.

remove libraries from the classpath as they are already available in a base image. This can be handled by the compileOnly or provided mechanisms?

The baseImage is more of a Docker thing that should not leak into the Gradle runtime classpath. By using compileOnly we would loose the ability to make use of it outside of the Docker setting. But this is maybe more of another story, created https://github.com/GoogleContainerTools/jib/issues/1436 a while ago. But being able to fiddle around with sourceSets, configurations and classpaths could provide a good entrypoint into such topics, in particular since so far there seems no consensus how to approach the problem in a generic manner in jib (there are more tickets related to layering). There are a variety of different possiblities in achieving this and nothing may fit every use case. The current lack of support is quite noticable in the layering/size of your images and registry.

But this are just a few examples where one may would like to have a bit more flexilbity. I can imagine all kind of other use cases. The Jar, JavaExec, War, Test and most of the Java classpath-related Gradle tasks allow to set the Configuration for this reason. For sure not the default use case, more part of those other 5%...

dmurat commented 5 years ago

I think that supporting custom source sets and configurations will be very useful. Otherwise, any Gradle project that uses these features cannot use jib. Exactly my case right now.

Please, provide these as jib's configuration options.

cromefire commented 4 years ago

This can be handled by the compileOnly or provided mechanisms?

A really important use case which at least sounds to be related is developmentOnly() for example (because it should put those deps into a separate configuration, namely developmentOnly). I have this case with a spring app right now, where jib just packs reactor-tools and spring-boot-devtools into the container, which clearly do not belong there and it's also pretty much impossible to extract it into a different module because it's just a conditional addition for development purposes.

It basically seems like JavaPlugin.RUNTIME_CLASSPATH_CONFIGURATION_NAME should be made configurable here (change it to runtimeElements for spring for example): https://github.com/GoogleContainerTools/jib/blob/27a949f63e61eb103dc47867e694c546c64d137f/jib-gradle-plugin/src/main/java/com/google/cloud/tools/jib/gradle/GradleProjectProperties.java#L193

I may look into making a PR for that if it's easy to pull off.

chanseokoh commented 4 years ago

@cromefire see the discussion in #2336 (which also includes a potential solution to the spring-boot-devtools issue). Basically, Maven/Gradle is the source of truth that defines which dependencies are required at runtime, and Jib just asks Maven/Gradle for the information. That said, it is the user's responsibility to set up Maven/Gradle dependencies correctly.

Just FYI, Jib has a general-purpose filtering extension (Maven / Gradle) that you can use to remove arbitrary files or move into other layers.

cromefire commented 4 years ago

Basically, Maven is the source of truth that defines which dependencies are required at runtime

Well I'm using gradle in this instance, which has a different system, so maven profiles won't help here. If anything, configurations seem to be a similar thing to maven profiles in the use case. I think you may be confusing gradle and maven here, as there are no profiles in gradle.

That said, it is the user's responsibility to set up Maven dependencies correctly.

Well they are properly set up in a configuration so they can be properly excluded, there's just the lack of an option to tell the plugin to use the proper "non-dev" configuration

Just FYI, Jib has a general-purpose filtering extension (Maven / Gradle) that you can use to remove arbitrary files or move into other layers.

While this would probably technically be working, it would be a PITA to filter the files your self, if there's just an easy fix for it.

chanseokoh commented 4 years ago

Sorry, I did mean to say Gradle instead of Maven. There's no difference between Gradle and Maven in this regard. (In #2336, you'll see that we have the Spring Boot extensions for both Maven and Gradle.)

There are multiple ways to have a profile-like behavior in Gradle. Perhaps an easy way is to use Gradle properties.

if (project.hasProperty('dev')) {

While this would probably technically be working, it would be a PITA to filter the files your self, if there's just an easy fix for it.

I agree using profiles is a much simpler way. I guess using a filter extension anyways requires you do use profiles. (BTW, just in case, the filtering extension is a different extension than the Spring Boot extension mentioned in #2336.)

chanseokoh commented 4 years ago

There are multiple ways to have a profile-like behavior in Gradle. Perhaps an easy way is to use Gradle properties.

Actually @loosebazooka has more expertise in dependency configuration in Gradle and will have some opinions about setting up conditional dependencies in Gradle.

cromefire commented 4 years ago

(In #2336, you'll see that we have the Spring Boot extensions for both Maven and Gradle.)

Okay I've missed that one, that seems to work for spring

There are multiple ways to have a profile-like behavior in Gradle. Perhaps an easy way is to use Gradle properties.

I don't see a point in point in duplicating things that are already in place here. From the gradle docs:

A Configuration represents a group of artifacts and their dependencies.

Which seems to pretty much what you'd want in this case and also allows you to be more flexible than with properties

cromefire commented 4 years ago

I may look into making a PR for that if it's easy to pull off.

I didn't really find the correct place where the dependencies are being resolved (yet, there's a lot of places where the project configurations and dependencies are mentioned), so sadly I wasn't able to get it working, so if somebody has a clue where that's happening, I can probably make it work

loosebazooka commented 4 years ago

Well gradle holds two different references to the runtime classpath. One as a Configuration and one as a property on a SourceSet. We use both of these to determine how to arrange the final artifact. You're going to have to modify both of those somehow in GradleProjectProperties.java.

The problem with this approach is that if you consider the gradle application plugin simliar to the jib plugin, they both act on the main sourceset and the default runtimeClasspath. It's not clear to me how much special handling needs to be done to handle non-standard builds.

Even configuring custom Jars seems to requires some sort of manipulation of the underlying CopySpec.

It would be helpful to show an example of how another packager handles this? And and example of how you think the configuration should be surfaced.

cromefire commented 4 years ago

Sure I can probably dig something up there but it'll probably take till the weekend sometime next week for me to get back to this

loosebazooka commented 4 years ago

What we really need is an example of someone achieving this via another packager, like an example using Jar (which probably uses the underlying archive/copyspec constructs) or an example using Application plugin.

For instance, how would you fill this out?

sourceSets {
  custom {
    // what goes here that will result in how the packager fulfills its directive?
  }
}

configurations {
  customImplementation.extendsFrom implementation
  customRuntime.extendsFrom runtime
}

I imagine for any packager to get this done correctly, either

  1. it would require some sort of source merging:
    sourceSets {
    custom {
      srcDirs sourceSets.main.java.srcDirs
    }
    }
  2. or some sort of multi sourceSet definition
    // somewhere in jib config
    sourceSets = ["main", "additional"]
cromefire commented 4 years ago

So some samples I came across are the license plugin: https://github.com/hierynomus/license-gradle-plugin/pull/184 And the shadow jar plugin also seems to allow configurations: https://imperceptiblethoughts.com/shadow/configuration/dependencies/

loosebazooka commented 4 years ago

I guess what I'm asking for is an example of how you've set up your configurations/sourcesets that would require this.

Jib can potentially accommodate to a point, but without a user correctly configuring things, we don't want to be a position of debugging someone's gradle build.

loosebazooka commented 4 years ago

@remmeier @dmurat can you share some small example build with multiple source sets and how you expect things to be configured?

cromefire commented 4 years ago

So basically I just have spring boot, where there is the extension, but that one only filters the dev tools and not all developmentOnly. To reproduce, basically start a new project at spring initializr and add something as developmentOnly()

chanseokoh commented 3 years ago

We are still waiting on someone to provide us with a concrete example (https://github.com/GoogleContainerTools/jib/issues/1778#issuecomment-710226417), as the team doesn't have a lot of Gradle expertise.

cromefire commented 3 years ago

Here's a quick demo: https://gitlab.com/cromefire_/jib-exclude-demo There are 3 ways of executing it:

You can Inspect the dependencies (in a specific configuration) via ./gradlew :dependencies [--configuration <configuration name>].

In this spring boot app specifically the are 2 configurations of interest:

Here's a scan so you can see it right in your browser: https://scans.gradle.com/s/zrexr57liwk2s/dependencies?toggled=W1swXSxbMCwxMDldLFswLDEwOF1d

cromefire commented 3 years ago

Here's also a similar case of a different plugin that also needed to implement configurable configurations because android (I know that doesn't apply here, but it's a similar case) uses a different configuration: https://github.com/hierynomus/license-gradle-plugin/pull/42

cromefire commented 3 years ago

The runtimeClasspath is hard-coded in 2 places: https://github.com/GoogleContainerTools/jib/search?l=Java&q=runtimeClasspath + RUNTIME_CLASSPATH_CONFIGURATION_NAME also contains "runtimeClasspath" Some time ago I tried to make that configurable, but I failed to get it working.

cromefire commented 3 years ago

I'd also say it's more a bug than a improvement, because this makes it impossible to use for some projects because there's simply no workaround available that I know or can think of.

chanseokoh commented 3 years ago

Thanks for the info. But it looks like the questions that @loosebazooka asked were rather more around the application plugin, multiple sourceSets, an example using Jar, etc. (See https://github.com/GoogleContainerTools/jib/issues/1778#issuecomment-694478069, https://github.com/GoogleContainerTools/jib/issues/1778#issuecomment-710226417, and https://github.com/GoogleContainerTools/jib/issues/1778#issuecomment-710228168). However, I don't really follow what @loosebazooka had in his mind, and unfortunately, he will not be available for a few months. If you can explain to me what the thoughts of @loosebazooka could have been, I'd appreciate it. And any design proposal for supporting this?

I'd also say it's more a bug than a improvement, because this makes it impossible to use for some projects because there's simply no workaround available that I know or can think of.

Anyways, reading your old comments and the example you just provided, I think your issue is specifically about Spring Boot and the custom developmentOnly that the plugin defines. At least you can use the Jib Layer-Filter extension (Maven / Gradle) to remove developmentOnly dependencies.

cromefire commented 3 years ago

I think your issue is specifically about Spring Boot and the custom developmentOnly that the plugin defines

Well spring boot is just a user of this and I think this is also where @loosebazooka is coming at you can do stuff like this:

configurations {
    register("docker") {
        extendsFrom(productionRuntimeClasspath.get())
    }
}

dependencies {
    "docker"("org.apache.commons:commons-lang3:3.11")
}

Which allows you to easily customize things.

cromefire commented 3 years ago

As of the sourceSets: I have no idea, how to do that, it's probably useful in some cases but those are probably really rare

cromefire commented 3 years ago

So I created a PR, which basically solves the problem with the configurations (see the sample project)

loosebazooka commented 3 years ago

From your example, I'm not convinced #3034 is the best solution, it's a little too convoluted.

I, however, can see the reason that someone would want this. Perhaps we just create a new jib configuration? So users can do something like

dependencies {
  jib "my.container-only:dependency:1.2.3"
}

and if a user wants to take advantage of the extendsFrom style workflows they can just call it on the jib configuration?

cromefire commented 3 years ago

Well your snippet is completely working with the PR you just have to create the jib configuration and tell jib to use it.

cromefire commented 3 years ago

So basically:

jib {
    configurationName.set("jib")
}

configurations {
    register("jib")
}

(Syntax might differ for the groovy version, I haven't checked that yet but it should work well because of the properties)

loosebazooka commented 3 years ago

Right, just forcing a convention here though.

cromefire commented 3 years ago

The problem with your jib by default proposal is compatibility. For that you'd have to have jib extend from runtimeClasspath by default and I'm not quite sure you can reverse the extending, so it defeats the whole purpose. In the case how I implemented it, the user has full control and there isn't any "hidden magic" happening.

cromefire commented 3 years ago

Right, just forcing a convention here though.

Well if you want a configuration you need to create a configuration, right? The only difference here is you explicitly opt in as opposed to something that the user might not want is happening in the background.

cromefire commented 3 years ago

It's basically saving 4 lines of code with your proposal while loosing all flexibility.

loosebazooka commented 3 years ago

So the idea wasn't to extend runtimeClasspath explicitly but instead add extra classpath options to the build.

productionRuntimeClasspath is a springboot only concept, and what wasn't clear is that spring-boot is polluting the runtimeClasspath with developmentOnly dependencies and then hacking around that when creating the production runtimeClasspath. This is not ideal and would have preferred their bootRun task to handle the developmentOnly configurations separately. Now it makes sense why you would require configuration selection rather than appending.

I'll take a look at the PR.

cromefire commented 3 years ago

I would imagine spring is not the only one doing this. But I've not looked at a lot of frameworks

chanseokoh commented 3 years ago

@remmeier @dmurat @saturnism @fhoeben @bric3 @atbentley @AndrewBentley6886 @ghilainm @cromefire

We've released Jib plugins 3.0 where you can set a custom Gradle Configuration. Check out the usage.

@cromefire thanks for your awesome contribution! Indeed, it was a highly requested feature.

chanseokoh commented 3 years ago

Fixed by #1776.

bric3 commented 3 years ago

@chanseokoh Thanks ! It is useful !