search

IQSS / dataverse

Open source research data repository software
http://dataverse.org
Other
883 stars 495 forks source link

Enable outgoing connections over HTTP/HTTPS proxy #4707

Closed blavoie closed 1 year ago

blavoie commented 6 years ago

By default dataverse is not able to connect to the internet over a proxy like Squid.

Under normal circumstances, using relevant system properties on the command line should make use of proxy effortlessly.

Parameters reference documentation: https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html

After investigation and discussion it appears that:

First EZID: by default doesn't support connections over HTTP proxy, a pull request for this part was sent to use system properties when present. It uses HttpClient and by default this library isn't using considering system properties until we explicitly call the [useSystemProperties()](http://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/impl/client/HttpClientBuilder.html#useSystemProperties()) on the HttpClientBuilder.

Second Dataverse is experiencing the same kind of problem when configuring harvesting to external Dataverse instances. To make HTTP calls Dataverse is using Unirest, that also uses HttpClient behind the scenes.

It is possible to quickly make Unirest to use an HttpProxy by two ways:

Don't know what is the best way of doing things in this case.

By doing a quick search I found that Unirest keyword is present in 8 files, not such a big deal to fix at first.

Thanks Bruno Lavoie

pdurbin commented 6 years ago

@blavoie thanks for opening this issue, for making the pull request at https://github.com/NCEAS/ezid/pull/11 , and for all the chatter on the dataverse-community mailing list!

If you are willing to make a pull request to address the Unirest issue, my initial thought is that the useSystemProperties approach is probably better but honestly, whichever one is easier for you to implement is probably fine. Thanks!

pdurbin commented 6 years ago

@blavoie hi! Any updates? Is this still an issue for you?

pdurbin commented 2 years ago

Dataverse is experiencing the same kind of problem when configuring harvesting to external Dataverse instances. To make HTTP calls Dataverse is using Unirest, that also uses HttpClient behind the scenes.

While it's true that Dataverse is using Unirest for the Data Capture Module (in DataCaptureModuleServiceBean.java for example), I don't believe it's used it harvesting (OAI-PMH).

So I'm removing the "Feature: Harvesting" label from this issue.

@blavoie by the way, are you still interested in this?

blavoie commented 1 year ago

@pdurbin not interested, can be closed. thanks