search

quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.57k stars 2.63k forks source link

Quarkus 1.4.2 Jib package issue while behind a corporate proxy #9682

Open agarcia-oss opened 4 years ago

agarcia-oss commented 4 years ago

Quarkus container-image-jib failed to download fabric8/java-alpine-openjdk11-jre base image through a http proxy I've being testing Quarkus jib and it doesn't seem to work behoind a proxy. My maven settings include the http_proxy config, but for some reasson the quarkus extension is not aware of it and it fails to download the fabric8 base image.

Expected behavior mvn package task should be able to make use of the user maven settings

Actual behavior mvn quarkus:add-extension -Dextensions="container-image-docker" mvn clean package -Dquarkus.container-image.build=true

[INFO] [io.quarkus.container.image.jib.deployment.JibProcessor] Starting container image build
[WARNING] [io.quarkus.container.image.jib.deployment.JibProcessor] Base image 'fabric8/java-alpine-openjdk11-jre' does not use a specific image digest - build may not be reproducible
[ERROR] [io.quarkus.container.image.jib.deployment.JibProcessor] I/O error for image [registry-1.docker.io/fabric8/java-alpine-openjdk11-jre]:
[ERROR] [io.quarkus.container.image.jib.deployment.JibProcessor]     org.apache.http.conn.ConnectTimeoutException
[ERROR] [io.quarkus.container.image.jib.deployment.JibProcessor]     Connect to registry-1.docker.io:443 [registry-1.docker.io/18.213.137.78, registry-1.docker.io/52.4.20.24, registry-1.docker.io/52.5.11.128, registry-1.docker.io/52.54.232.21, registry-1.docker.io/54.236.131.166, registry-1.docker.io/52.72.232.213, registry-1.docker.io/54.85.107.53, registry-1.docker.io/107.23.149.57] failed: connect timed out
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

The package process works just fine in my same enviroment when I don't use my corporate proxy. The extension documentation doesn't seem to have any http proxy config parameter:

https://quarkus.io/guides/container-image#jib-options

Configuration My settings file. 

  <proxies>
    <proxy>
      <id>http</id>
      <active>true</active>
      <protocol>http</protocol>
      <host>10.113.55.36</host>
      <port>8080</port>
    </proxy>

    <proxy>
      <id>https</id>
      <active>true</active>
      <protocol>https</protocol>
      <host>10.113.55.36</host>
      <port>8080</port>
    </proxy>
  </proxies>

Environment (please complete the following information):

geoand commented 4 years ago

Thanks for reporting.

The jib extension doesn't currently look into your maven settings to fetch such properties. So as a workaround you can set the necessary settings using something like 

mvn clean package -Dquarkus.container-image.build=true -Dhttp.proxyHost=10.113.55.36 ....

@aloubyansky I assume bootstrap already handles Maven Settings files correctly. Can I from an extension pull out that information? Would it make sense to make it available as a build item?

aloubyansky commented 4 years ago

Not atm. You also shouldn't assume the build is driven by Maven. If there is a requirement to have access to a networking config we should create an issue for it. A simple way of getting access to the Maven settings would be

new BootstrapMavenContext().getEffectiveSettings()
geoand commented 4 years ago

I agree, I'm just trying to collect all options here.

Thanks for the info @aloubyansky

geoand commented 4 years ago

Changing to enhancement because this isn't an actual bug.

It would be nice to have a general way of handling such cases, but given how dynamic Gradle can be, I don't think we can solve anything other than the Maven case. Furthermore, there might also be other extensions that need to make network calls that would benefit from this same type of thing.

I wonder if it makes sense to have some kind of way of setting proxy fields before a build step is executed and restoring the old values when it's done. This could be driven by a flag perhaps. Or maybe these settings should be applied for the whole build, in which case we would just set them before the build starts. But again we need a way to know these proxy settings. Maybe we could have a Quarkus build time Config object that we populate only for the case of Maven (for Gradle we could tell users to use gradle.properties for example).

WDYT @aloubyansky?

aloubyansky commented 4 years ago

I'd probably prefer providing this kind of info on demand instead of looking it up and populating some config objects just in case some one needs them. I think we could have some kind of Build[System]Context that would be a facade for providing various build-related settings that could be discovered and provided lazily on demand. It could be set on QuarkusBootstrap.Builder that our Maven mojos and Gradle tasks call to initiate the build. It's not as straightforward for the devmode though, since it's kind of isolated in a different process.

geoand commented 4 years ago

Even in devmode though, we could pass that info to the devmodecontext, righ?

aloubyansky commented 4 years ago

If you want to do it now and if we want to support gradle, we'd have to serialize the info to pass to the dev process. Eventually we are going to introduce custom models in our gradle plugin that will provide app deps and could probably provide this kind of info too. That should be discoverable in dev mode, IDEs, etc.

geoand commented 4 years ago

Given that this is the only such case we have so far and that currently container-image only works for the prod-mode, I don't think we need to complicate dev-mode (until we actually have a use case).

agarcia-oss commented 4 years ago

Just a quick update, the workarround of setting up the https proxy config from the command line works fine:

mvn package -Dquarkus.container-image.build=true -Dhttps.proxyHost=10.113.55.36 -Dhttps.proxyPort=8080

[INFO] [io.quarkus.container.image.jib.deployment.JibProcessor] Using base image with digest: sha256:f59f2dfb8dc75e6fbdb3b5661ab88a3a9de960ca8e13ec478f796a6feb62e990
[INFO] [io.quarkus.container.image.jib.deployment.JibProcessor] Container entrypoint set to [java, -Dquarkus.http.host=0.0.0.0, -Djava.util.logging.manager=org.jboss.logmanager.LogManager, -cp, /app/resources:/app/classes:/app/libs/*, io.quarkus.runner.GeneratedMain] 
[INFO] [io.quarkus.container.image.jib.deployment.JibProcessor] Created container image quarkus:0.0.1-SNAPSHOT (sha256:91b248ba31afac2b3e91c12c22f020f21957bfb76181ce31e5f108318d9a1169)

[INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 20605ms
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  39.369 s
[INFO] Finished at: 2020-06-01T08:45:37+02:00
[INFO] ------------------------------------------------------------------------

Thanks for the feedback!

geoand commented 4 years ago

Given that

1) adding new BootstrapMavenContext().getEffectiveSettings(); into a build step would be rather unclean 2) the workaround is rather simple

I propose we leave this open for now and act on it if lots of folks report it being an issue.

Another thing I thought of is that we could set system properties for the proxy that apply throughout the build based on what is in proxy. But again, that's probably something we only want to look into if this becomes an issue.

aloubyansky commented 4 years ago

We can surely set the properties. It seems like in Gradle this is how it's done anyway. And in Maven we can simply set them in Mojos by reading the settings.

geoand commented 4 years ago

Exactly, that's what gradle does when user set systemProps.http.proxyHost=something in gradle.properties.

@aloubyansky do you think setting the properties in the mojos is something we should do now or something we should wait for user demand?

It seems like a reasonable and innocent thing to do :)

aloubyansky commented 4 years ago

I wouldn't do it just because we can. I thought JibProcessor or something was missing this info. If it's not the case then I'd wait for a request.

aloubyansky commented 4 years ago

My idea was simple: we could provide some NetworkingConfigBuildItem that would expose methods to get proxy settings. But it would resolve them when those methods are actually invoked. In Gradle it would be based on properties. In Maven it would be based on properties + settings. But I would still do that on request.

geoand commented 4 years ago

OK yeah. Jib could use this, but since it currently seems to be just an edge case, let's wait and see if more users will request it.

vaishnav-piyush commented 1 year ago

I am facing similar issue with gradle. I am using the gradle quarkus plugin and jib to build and publish my image.

./gradlew assemble -Dquarkus.container-image.build=true -Dquarkus.container-image.push=true -Dquarkus.container-image.image=<image> -Dquarkus.container-image.username=<image-repo-user> -Dquarkus.container-image.password=<image-repo-password> -Dhttp.proxyHost=<http-proxy-host> -Dhttp.proxyPort=<port> -Dhttps.proxyHost=<https-proxy-host> -Dhttps.proxyPort=<port>

The image repository is only accessible via a corporate proxy.

The issue stated when I first upgraded to Quarkus 3. I have tried with different versions of Quarkus.

Quarkus Result
3.2.0.Final ✖
3.1.2.Final ✖
3.1.1.Final ✖
3.0.2.Final ✖

I am using Java 11.0.6 and Gradle 8.2.

geoand commented 1 year ago

The issue stated when I first upgraded to Quarkus 3

So it worked with Quarkus 2.16?

vaishnav-piyush commented 1 year ago

So it worked with Quarkus 2.16?

I have it working in another app which is on Quarkus 2.14.1.Final, haven't tried with 2.16.

dprinz commented 8 months ago

Any updates on this. I still have this problem with Quarkus 3.6.6 and Gradle 8.5

dprinz commented 8 months ago

Can this be classified as a bug? With Quarkus 3 and Gradle, the jib seems to be unusable behind a proxy.

geoand commented 8 months ago

Done

geoand commented 8 months ago

My idea was simple: we could provide some NetworkingConfigBuildItem that would expose methods to get proxy settings. But it would resolve them when those methods are actually invoked. In Gradle it would be based on properties. In Maven it would be based on properties + settings. But I would still do that on request.

@aloubyansky I wonder if it's time for this :)

dprinz commented 8 months ago

Not sure if this is the same bug, but it's also impossible to pass a custom truststore to jib. Setting -Djavax.net.ssl.trustStore=cacerts is completely ignored as is -Dhttps.proxyHost=