search

eclipse / ecf

ECF project repository
7 stars 14 forks source link

org.eclipse.ecf.provider.filetransfer.httpclientjava does not support proxy authentication #76

Open merks opened 11 months ago

merks commented 11 months ago

See the following p2 issue for background details:

https://github.com/eclipse-equinox/p2/issues/381

It's not clear (to me) if the proxy needs to be configured by java.net.http.HttpClient.Builder.proxy(ProxySelector) as suggested here:

https://openjdk.org/groups/net/httpclient/intro.html

Note that with org.eclipse.ecf.internal.provider.filetransfer.httpclientjava.Activator.USE_SHARED_CLIENT_DEFAULT being true, I think there is only a singe HttpClient that is shared.

These two fields

https://github.com/eclipse/ecf/blob/bbf5ffad9ac736bf6f7eb6690f16c1ba0dfa588e/providers/bundles/org.eclipse.ecf.provider.filetransfer.httpclientjava/src/org/eclipse/ecf/internal/provider/filetransfer/httpclientjava/ECFHttpClientFactory.java#L35-L38

which are copied from here:

https://github.com/eclipse/ecf/blob/bbf5ffad9ac736bf6f7eb6690f16c1ba0dfa588e/providers/bundles/org.eclipse.ecf.provider.filetransfer.httpclient5/src/org/eclipse/ecf/internal/provider/filetransfer/httpclient5/ECFHttpClientFactory.java#L62-L63

are not actually used. It's not clear what, if anything, needs to be configured in this regard.

The following implementation detail is clearly incomplete:

https://github.com/eclipse/ecf/blob/bbf5ffad9ac736bf6f7eb6690f16c1ba0dfa588e/providers/bundles/org.eclipse.ecf.provider.filetransfer.httpclientjava/src/org/eclipse/ecf/internal/provider/filetransfer/httpclientjava/HttpClientProxyCredentialProvider.java#L77

Perhaps it would be correctly implemented like this:

Credentials credentials = getCredentials(new AuthScope(getRequestingHost(), getRequestingPort()));

It's possible that if this were fixed, some additional proxy configurations would "just work"...

scottslewis commented 11 months ago
> The following implementation detail is clearly incomplete: > > https://github.com/eclipse/ecf/blob/bbf5ffad9ac736bf6f7eb6690f16c1ba0dfa588e/providers/bundles/org.eclipse.ecf.provider.filetransfer.httpclientjava/src/org/eclipse/ecf/internal/provider/filetransfer/httpclientjava/HttpClientProxyCredentialProvider.java#L77 > > Perhaps it would be correctly implemented like this: > > ``` > Credentials credentials = getCredentials(new AuthScope(getRequestingHost(), getRequestingPort())); > ``` > > It's possible that if this were fixed, some additional proxy configurations would "just work"... I agree with @merks that this might fix some (or even all) of the observed proxy creds regressions. I have no way of testing whether it will help or not however, so we should line up some testing assistance and/or environment before making this fix.
SebasHein commented 11 months ago

I am one of the users behind an authentification proxy. I know you would prefere some kind of unit test work environment but thats apparently hard to set up.

If you could provide me with a build of the eclipse sdk that contains the fix @merks suggests, I could do a test and see wether it works again - if that is of any help of course.

merks commented 11 months ago

Yes, my sense is that the fix can do no harm. In fact the called getCredentials method looks to me highly likely to produce NPEs when passed null as it current does so I feel we should change this regardless of there not being good automated tests...

@SebasHein

Locally for testing this technology stack, I use the installer with these projects for the workspace:

image

That lets me change and test everything (ECF, p2, and Oomph) locally.

If you wish to help with testing such a change before there is an ECF build, let's discuss that further in this Oomph issue that I just opened:

https://github.com/eclipse-oomph/oomph/issues/46

Thanks again for reporting the issue...

merks commented 11 months ago

@SebasHein

I produced this update site that you could use for testing the potential fix:

https://ci.eclipse.org/emf/job/test-ecf/lastSuccessfulBuild/artifact/releng/org.eclipse.ecf.releng.repository/target/repository/

sratz commented 11 months ago

I have looked into this in more detail and have come to the conclusion that the implementation is incomplete regarding proxy authentication.

The whole code base of java 11 client was copied from apache http client 5 and adapted.

However, apache 5 worked significantly different:

The Java 11 HTTP client however, does not allow request-specific credential providers or proxy selectors. It only allows a global one set on the HttpClient instance during its construction.

The current ECF wrapper for Java 11 tries to imitate such a 'request context' object by introducing a custom interface https://github.com/eclipse/ecf/blob/3485821a7c5c8948d8a82caca5a454a04c45607d/providers/bundles/org.eclipse.ecf.provider.filetransfer.httpclientjava/src/org/eclipse/ecf/internal/provider/filetransfer/httpclientjava/IHttpClientContext.java

and the proxy and credential information goes to the implementation of that interface: https://github.com/eclipse/ecf/blob/3485821a7c5c8948d8a82caca5a454a04c45607d/providers/bundles/org.eclipse.ecf.provider.filetransfer.httpclientjava/src/org/eclipse/ecf/internal/provider/filetransfer/httpclientjava/ECFHttpClientFactory.java#L64-L91

The problem is - as can be seen by the @SuppressWarnings("unused") - this information is simply going nowhere, because there is no notion of such a client request context in the Java 11 HTTP client.

I don't think there is an easy way around this.

Two options have come to mind:

  1. Somehow hack in this per-request-context by passing along the information in some static Map / via ThreadLocals. The problem is that the Java 11 HTTP client is async by default and we are not in the same thread in HttpClientFileSystemBrowser and in the ProxySelector / Authenticator. So there is no key for a mapping of request -> IHttpClientContext that we could leverage for a lookup.
  2. Do not re-use the HTTP client instances at all. Instead, create a fresh one for every ECF call. This probably results in worse performance, as sockets / caches cannot be shared and re-used. It also would change the public API of org.eclipse.ecf.internal.provider.filetransfer.httpclientjava / org.eclipse.ecf.provider.filetransfer.httpclientjava significantly.
sratz commented 11 months ago

Another thing:

The Java HTTP client - by default - disallows Basic Authentication for proxies since Java 8u111:

So after solving the problem mentioned above, we would - additionally - need the following system properties to be set:

jdk.http.auth.proxying.disabledSchemes = ""
jdk.http.auth.tunneling.disabledSchemes = ""

to get things working. This would need to happen probably via eclipse.ini, as these are initialized statically very early: https://github.com/openjdk/jdk17u/blob/df82c1c6490bb5114fe52e4e1a4faa6aa3a56cfd/src/java.net.http/share/classes/jdk/internal/net/http/common/Utils.java#L197-L216 so a System.setProperty() would probably be too late.

laeubi commented 11 months ago

@sratz Prox Auth is done on the http level, so one always could set these headers manually (we do that at Tycho already)...

merks commented 11 months ago

@scottslewis

Note that final platform build is on Wednesday so it will be good to have a final repository URL for that:

https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/issues/1390

Also for contribution to SimRel. If you let me know the URL I will do that for ECF.

Thanks.

scottslewis commented 11 months ago

@merks

Here is release build repo for ECF 3.14.40:

https://download.eclipse.org/rt/ecf/3.14.40/site.p2/3.14.40.v20231114-1017/

scottslewis commented 5 months ago

@merks @laeubi

As @laeubi appears disinclined to implement this (he is the implementer of the httpclientjava provider). I'm going to close this issue. If it should be reopened then please do so.

jonahgraham commented 4 months ago

Can you consider reopening this for planning because we have started a funded dev effort initiative to cover this issue.

laeubi commented 4 months ago

This is somehow related:

jonahgraham commented 4 months ago

There seems to be an open question as to which approach for handling http going forward should happen. The ECF and Equinox/p2 project teams need to decide on the correct course of action so that the planning council can recommend funding this work to the IDE WG. (See this comment in gitlab)

merks commented 3 months ago

This is a related problem that I too encounter only with the newer provider:

https://github.com/eclipse-equinox/p2/issues/529