Support authenticated Proxies

[u3r]

Hi Michel,

my company just introduced a new authenticated proxy and half of all build tools went to hell: Sadly the download task is one of those not working.

I tried to create a minimal examle: This buildfile downloads the gradle-download-task (through the proxy - working for gradle 2.12 - not working for gradle 2.7) but fails during the download task.

// build.gradle
buildscript {
    repositories {
        mavenCentral()    
    }
}

plugins {
    id "de.undercouch.download" version "2.1.0"
}

import de.undercouch.gradle.tasks.download.Download

ant.'nexus-version'='2.9.1'

repositories {
    mavenCentral()    
}

task downloadNexus(type: Download) {
    src( "https://download.sonatype.com/nexus/oss/nexus-${ant.'nexus-version'}.war")
    new File(buildDir, "downloads/nexus-${ant.'nexus-version'}.war")
    dest "${buildDir}/target/webapp/nexus.war"
}

#gradle.properties
# not working
systemProp.http.proxyHost=ourProxySupportingBasicAuth
systemProp.http.proxyPort=8080
systemProp.http.proxyUser=user
systemProp.http.proxyPassword=password
systemProp.http.nonProxyHosts=localhost|*.some-domain.com

systemProp.https.proxyHost=ourProxySupportingBasicAuth
systemProp.https.proxyPort=8080
systemProp.https.proxyUser=user
systemProp.https.proxyPassword=password
systemProp.https.nonProxyHosts=localhost|*.some-domain.com

# cnltm configuration - cntlm forwards to our proxy but takes over authentication - works flawlessly
# even for gradle --debug downloadNexus
#systemProp.http.proxyHost=localhost
#systemProp.http.proxyPort=3128

#systemProp.https.proxyHost=localhost
#systemProp.https.proxyPort=3128

Here is the (slightly redacted) log of gradlew --refresh-dependencies --debug downloadNexus Attaching the log is currently not working for me. I'll send you an email.

I hope this is useful.

Thanks in advance

[michel-kraemer]

Your configuration looks OK to me.

In your log files I can see the credentials that you are using to connect to the proxy server (except the password of course). They look like dummy data (i.e. proxyUser and company.com). Please check if your username/password and proxy server are really correct for your company.

If everything is correct, it would be interesting to know if it's a problem with gradle-download-task or with Java or Gradle in general. If you call gradlew --refresh-dependencies it should download all dependencies through the configured proxy server. Please add a dummy dependency to your minimal example to see if it is downloaded correctly.

Cheers, Michel

[u3r]

Hi - I did that before submitting the bug but removed it from the example. A test dependency (junit from maven central) as well as the download task itself load allright through the proxy. I did not have a look inside your code yet but I know java itself only does unauthenticated proxy - you have to submit the authentification yourself. I don't know if gradle does that automatically for plugins (or at least provides an api)

[u3r]

Hi Michel,

i played around with this problem and would like your opinion on some of the options I see:

option one

One could "fix" this by installing Authenticator.setDefault( new Authenticator() {...} ) to inject the proxy password/user. However I don't like it since it is a singleton potentially influencing all plugins within the build. It seems to be the only way to alter the behaviour of URLConnection - no other way to provide credentials (safe option two)

option two

It is possible to add Proxy-Authorization as requestProperty (Base64 encoded). This works for http targets but fails for https. For https one would need to first establish connection to the proxy, then establish ssl connection and then access the target. This seems a lot of work to duplicate but would not require a new dependency (I started this but did not get it to work yet)

option three

Use a library - I tested with org.apache.httpcomponents:4.5.2: this works well, is relatively clean code but has a very different api from URLConnection.and introduces a new dependency. But it allows setting a credentialsprovider only locally for one/some connection(s).

What do you think?

[michel-kraemer]

@u3r Thanks for describing the possibilities so carefully. I just pushed a commit that adds support for (authenticated) proxy servers. If you like you can build the library yourself and test it.

I chose option one in this case. From my point of view this is OK although Authenticator is a singleton. However, build processes are usually short-lived and setting a global default Authenticator should not affect anything else. If it does we might as well reset Authenticator to the previous one after the file has been downloaded. This should not be too hard to implement.

[u3r]

Hi Michel,

the following is not meant as criticism and I greatly appreciate your fast responses despite me not offering a pull request: You probably you know gradle better than me but my reasoning for disliking option 1 is as follows: Unless gradle separates plugins in an osgi-like manner the authenticator is shared between all plugins. Additionally the authenticator is used/can be used for all kinds of authentication (including password protected target urls - see Oracle-Doc ) - so substituting a non-delegating authenticator that only returns the proxy password (and in case of your commit returns the proxy password for any auth-query) is asking for very hard to debug problems due to plugin-order. Regarding your idea about resetting the authenticator: how does gradle handle its parallel tasks? I doubt it spawns new JVMs so that would be yet another race-condition waiting to happen.

All in all I think the authenticator approach is only a good solution for stand alone java-apps.

[michel-kraemer]

You're totally right and I share your concerns. The only way to get to a clean solution would be to use Apache HttpClient or something similar. I don't know when I can implement this. I hope in one or two weeks. I hope that's OK for you. I'll leave this issue open. Consider the Authenticator solution a temporary one and nothing more than a workaround.

[michel-kraemer]

By the way: Gradle already depends on Apache HttpClient, so we don't introduce anything new here. I just hope that the version they use in Gradle 1.x is compatible to the one they use now. Otherwise we might have to drop support for Gradle 1.x -- which would not be a bad thing anyhow.

[u3r]

I'm not in a hurry - I used the Authenticator.setDefault workaround within the gradle script (yay gradle).

[michel-kraemer]

I did some tests and it seems we can migrate to Apache HttpClient without problems. However I need to increase test coverage first so I can make sure the plugin still works reliably after the refactoring. I'll keep you up to date.

[michel-kraemer]

I refactored the plugin. It uses Apache HttpClient now. Support for authenticating proxies should work as well.

May I ask you to test the plugin from the master branch? Thanks!

Michel

[u3r]

Will do - might take me till end of next week though.

Thanx for the effort!

[michel-kraemer]

Thanks for your support. I pushed a snapshot to make it easier for you to test the new version:

https://oss.sonatype.org/content/repositories/snapshots/de/undercouch/gradle-download-task/3.0.0-SNAPSHOT/

Let me know if it works or not. Thanks!

[u3r]

Hi Michel,

I tested again today:

• [x] Negative test: 2.1 still fails with my companies proxy
• [x] Positive test: same proxy settings succeed with 3.0.0-SNAPSHOT

Thank you for the fix!

[michel-kraemer]

Thank you so much! I'll release the new version today.