system properties not applied? or too late?

[childnode]

I tried to define the proxy settings via an environment, so I created gradle-proxy.properties and set

org.gradle.project.systemProp.http.proxyHost=proxy.example.com
org.gradle.project.systemProp.http.proxyPort=8080
org.gradle.project.systemProp.https.proxyHost=proxy.example.com
org.gradle.project.systemProp.https.proxyPort=8080
org.gradle.project.systemProp.http.nonProxyHosts=localhost
#org.gradle.project.systemProp.http.auth.ntlm.domain=sso.example.com

while ./gradlew -c gradle.init.bkuProxy.properties --refresh-dependencies build works

so then, my first thought was: ok, then the system properties are applied too late, the plugin might not be able to add them on init time so any network resolves will conquer it?

But then I read about this: http://stackoverflow.com/a/33636645/529977

so: do we have a bug, a stupid user oder just a "works as expected" here? In any case, I would appreciate some warm words of explanation ;)

With kind regards, ~Marcel

[stevesaliman]

I'm not sure why ./gradlew -c gradle.init.bkuProxy.properties --refresh-dependencies build works, but I think I know why ../gradlew -PenvironmentName=proxy build doesn't

Setting a property in a property file, like gradle.properties is done with something like this: systemProp.http.proxyHost, which works if it is in gradle.properties.

The second problem is that Gradle seems to be doing something different when it encounters something that starts with systemProp, which the plugin doesn't. This is a bug in the plugin, which I'll try to fix this weekend.

In the meantime, a workaround would be to set regular properties in your gradle-proxy.properties file, then have code in build.gradle that looks for the existence of these properties and manually sets the system property accordingly.

Thank you for finding this bug,

Steve

[childnode]

sorry, it was too late last night for me, the -c .properties option indeed doesn't work .. neither it is the correct format for -c ... but never mind ..

thank you for the clarification and ahead thank you for fixing this ;)

~Marcel

P.S: what does you do more than applying an additional properties file? And why it is different to "gradle only" gradle.properties definition where systemProp.http.proxyHost is enough but in environment properties you have to set org.gradle.project.systemProp.http.proxyHost ? I would think systemProp is already a uniq, well defined prefix for setting all system properties, so why the full-qualified one especially for this plugin?

[stevesaliman]

When gradle starts up, it reads certain files as described in the readme for this project. This means that anything defined in those files will already be defined at the time the properties plugin is applied. This includes setting System properties if file contents start with the right prefix.

The properties plugin starts from scratch, re-reading the standard files, but with a couple more files thrown in, as described in the readme. The intended result is for properties to be re-defined based on the rules of the plugin. But the plugin doesn't try to set system properties, hence the bug you found.

As for the difference between systemProp.http.proxyHost and org.gradle.project.http.proxyHost, I think that one is meant for use in property files, and the other is meant for the command line. I'm basing that on https://docs.gradle.org/current/userguide/build_environment.html. Also note that the command line version takes org.gradle.project.myvar not org.gradle.project.systemProp.myvar

[stevesaliman]

I started looking into this further, and I think I can offer some clarification.

There is a basic difference between the org.gradle.project prefix and the systemProp syntax. One is used for reading system properties, the other is used for writing them.

The org.gradle.project prefix can be added to a Java system property to tell Gradle that the system property should also become a gradle property. So running gradle with gradle -Dorg.gradle.project.myProperty=myValue would set a Java property (note the -D instead of -P), and gradle would then create a build property called myProperty. This is handy when you can specify Java properties during a build, but you can't edit property files. This is not you need.

The systemProp prefix is for the opposite direction. It tells Gradle to take a build property and add it to the Java system properties as part of the build. So putting systemProp.http.host=myhost in one of the property files tells Gradle to set the Java system property http.host to the value myhost before it executes tasks. This is the one you want. I'm looking at making the plugin do this for you when the properties in question are in an "environment" file like gradle-proxy.properties.

[stevesaliman]

I just released version 1.4.5-SNAPSHOT to the Maven Central Snapshot repository. Please give it a try and let me know how it works for you. The new version should be found if you have the following in your build.gradle file:

buildscript {
  repositories {
    maven { url "http://oss.sonatype.org/content/repositories/snapshots" }
  }
}

This new version should enable you to set your http proxy variables by adding the following to your gradle-proxy.properties file.

systemProp.http.proxyHost=proxy.example.com
systemProp.http.proxyPort=8080
systemProp.https.proxyHost=proxy.example.com
systemProp.https.proxyPort=8080
systemProp.http.nonProxyHosts=localhost

There are two important things to keep in mind:

1. If your project is part of a multi-project build, you must put the values above in a file that is in the root project's directory. Gradle does NOT set system properties based on values found in child projects.
2. You have to run specify the right environment name on the command line like this: ./gradlew -PenvironmentName=proxy build

[childnode]

hi again,

first of all: 1.4.5-SNAPSHOT DOESN'T work for me as expected ;(

but first one question regarding your comments above: if any org.gradle.project prefix'ed property is converted into a system and project extra property, then a valid alternative would be to set:

org.gradle.project.http.proxyHost=proxy.example.com
org.gradle.project.http.proxyPort=8080
org.gradle.project.https.proxyHost=proxy.example.com
org.gradle.project.https.proxyPort=8080
org.gradle.project.http.nonProxyHosts=localhost

why do I get a same called extra property afterwards setting these in an environment? expected to get a gradle property http.proxyHost but got org.gradle.project.http.proxyHost. testing with

    help {
        doLast {
            project.extensions.extraProperties.properties.sort().each { p ->
                println ' + ' + p.key + " = " + p.value
            }
        }
    }

but you can also see this with gradle properties.

Expected: systemProp. properties are not applied to the gradle project. but org.gradle.project. are without their prefix. Which leads me to:

taking note of your comment in the added readme section https://github.com/stevesaliman/gradle-properties-plugin/commit/07fdb978 and https://docs.gradle.org/current/userguide/build_environment.html it would be consistent to handle org.gradle.project. properties too, else it would be another trap.

Back to why it doesn't work for me: While it is set correctly ( as implemented in https://github.com/stevesaliman/gradle-properties-plugin/commit/07fdb978#diff-981f2f5cb70a361ce10538147b598928R348 ) gradle not uses the system property correctly ... why: don't know:

setting the

systemProp.http.proxyHost=some.foo

in the gradle.properties, then ./gradlew --info --refresh-dependencies help it works as expected, you'll get s.th. like

* What went wrong:
Error resolving plugin [id: 'org.sonarqube', version: '1.2']
> Could not GET 'https://plugins.gradle.org/api/gradle/2.11/plugin/use/org.sonarqube/1.2'.
   > some.foo: unknown error

setting them in an env file and calling ./gradlew --info --refresh-dependencies -PenvironmentName=proxy help (or setting environmentName=proxy in the gradle.properties) gradle doesn't use the proxy.

Hint: it also doesn't work for ./gradlew -PgradleUserName=proxy and a set up ~/.gradle/gradle-proxy.properties. same as for an environment. Works different to gradle.properties

:warning: if it turns out, it's too malicious to re-implement what gradle do, it would be fine for me to just take a note on it in the readme that system properties are just "not allowed" in any extra property file!!

:question: another question, why do you extra add gradle.properties on your own code since this is the gradle default as you describe in the readme?

• buildPropertyFileListFromSettings https://github.com/stevesaliman/gradle-properties-plugin/commit/07fdb978#diff-981f2f5cb70a361ce10538147b598928R297
• buildPropertyFileListFromProject https://github.com/stevesaliman/gradle-properties-plugin/commit/07fdb978#diff-981f2f5cb70a361ce10538147b598928R268 both use different orders too?!

P.S: ./gradlew --info -Dorg.gradle.project.http.proxyHost=foo.bar help doesn't override system property systemProp.http.proxy=proxy.example.com set in ~/.gradle/gradle-proxy.properties the resulting system property set is still proxy.example.com so org.gradle.project.http.proxyHost seems not to be a valid gradle property conversion to system property http.proxy while project."http.proxy" is set from commandline. it's getting weired ...

P.P.S: why it doesn't work for networking properties would be reasonable to me, when gradle sets up the network prior any other tasks. so setting the system property afterwards as you did for systemProp.http.proxy it doesn't affect the already set up connection manager .. but just an assumption

[stevesaliman]

The key to the whole mystery was revealed when I started researching your very last question. I wish I had read it first :-) I now realize that you're trying to configure Gradle itself, and not any particular task in your build.

It looks like we're dealing with built in Gradle functionality, and I think there may be a bug there. Gradle reads the standard files before it does anything else, which is why it all works when you put the properties in a standard location, like <project_root>/gradle.properties or ~/.gradle/gradle.properties. Unfortunately, Gradle appears to set up the proxy before it reads your build.gradle file. The documentation says that you can set the http.proxyHost in build.gradle with System.setProperty('http.proxyHost', 'some.bar'), but it doesn't work. When I tried it, it doesn't complain that some.bar is invalid, it complains that some.foo is invalid because that is what was in my gradle.properties file.

I'll do some more digging, and raise this with the Gradle team to see if I'm missing something, but it sort of makes sense. You can't apply a plugin before the plugin's jar has been downloaded, and Gradle needs to have a working proxy before you can do that. The bug may be the in the documentation and not the code.

If my hunch is correct, you can use the properties plugin to set any build or system properties as long as they aren't used by Gradle itself before it configures your build.

The rest of this post attempts to answer the other questions you had in your last response.

Build properties starting with org.gradle.project are not converted into Java System properties. Java System properties that start with org.gradle.project are converted into build properties. Note the -D instead of the -P in the example given in the Gradle docs. You can have a build property that starts with org.gradle.project, but it will be just another build property. It will not be added to Java System properties. This also explains the output you see in your test.

If you start a build property with systemProp., Gradle will set a Java system property with whatever came after the ., but it won't change the build property at all.

Based on that, I think the README file is what it needs to be. Gradle doesn't do anything to build properties starting with org.gradle.project, so the plugin doesn't either.

When the plugin processes files, it processes the gradle.properties file again because gradle processes the standard files before any plugins are applied, so I process them all again to make sure the correct order of precedence is followed. The order of processing is different in the two methods you mention because gradle does things in a different order depending on if the plugin is applied in a project (in build.gradle), or in settings (in settings.gradle).

System properties can be used to set build properties, not other system properties. Therefore -Dorg.gradle.project.http.proxyHost will create a build property named http.proxyHost, but it does nothing to system properties.

I hope this helps,

Steve

[stevesaliman]

I just released version 1.4.5 of the plugin which now sets System properties correctly.

There isn't much I can do for this particular http proxy problem because we have a chicken and egg problem. We need the http proxy set to refresh the plugin soucres, but we need the plugin to get system properties from a non standard file.

[childnode]

yes, that's reasonable behaviour ... thx for fixing it for common system properties we need i.e. for run or tests :+1: