Support an upstream SOCKS proxy

[JeremyRand]

Supporting an upstream SOCKS proxy would be very beneficial to users who want to use YaCy with Tor. One particular requirement to keep in mind is that Tor's stream isolation feature (which is very important for privacy in a situation like YaCy's) requires SOCKS authentication.

I've spent a few hours looking around, and the best candidate Java SOCKS client library I can find that supports authentication is https://github.com/fengyouchao/sockslib .

I'm not 100% sure that sockslib is the best choice (I'd probably want to ask some other Tor community members if there's something else they'd recommend), but before I expend any additional effort on this, would there be interest among the YaCy devs to support an upstream SOCKS proxy via sockslib? If so, would anyone like to do the work of integrating sockslib into YaCy, or should I attempt it myself and submit a PR?

[luccioman]

Hi @JeremyRand , wouldn't it be possible to get what you want with the JDK Socks support itself and eventually some Authenticator configuration?

[smokingwheels]

outbind://18-0000000022B579CC8EC80B459CF8644B326A5F2004E12200/ IE chucked that msg at me when I opened this one.. -----Original Message----- From: luccioman [mailto:notifications@github.com] Sent: Wednesday, October 19, 2016 4:23 PM To: yacy/yacy_search_server Subject: Re: [yacy/yacy_search_server] Support an upstream SOCKS proxy (#84)

Hi @JeremyRand , wouldn't it be possible to get what you want with the JDK Socks support itself and eventually some Authenticator configuration?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[luccioman]

Hi @smokingwheels , mmh I don't understand... the links in my previous comment makes your IE angry?

[smokingwheels]

Yes it is stuck in my outbox for full details..All I said was

Yes Sets hells bells ringing a security problem or somthing when I opened this on I got outbind://30-0000000022B579CC8EC80B459CF8644B326A5F2084E12200/

[luccioman]

Strange, these two links are hosted on the not so exotic docs.oracle.com domain...

[JeremyRand]

Hi @luccioman ,

Hi @JeremyRand , wouldn't it be possible to get what you want with the JDK Socks support itself and eventually some Authenticator configuration?

As far as I can tell, using Authenticator allows a global username/password to be set per host. I'm not sure whether this host is unique per SOCKS proxy or per host being accessed through the SOCKS proxy, but what it does not allow (from what I understand) is using different authentication data for different TCP connections that are to the same host via the same proxy. Unfortunately, that use case is what's needed for Tor's stream isolation.

For example, let's say that Alice searches YaCy for "WikiLeaks submit", and then searches YaCy for "Chinese restaurant Salt Lake City". Alice probably doesn't want to reveal that someone located in Salt Lake City who went to a Chinese restaurant that night is planning to submit something to WikiLeaks. Therefore, those searches should go over different Tor circuits. Since those searches are likely to go to the same YaCy nodes over the same SOCKS proxy, the JDK's Authenticator isn't safe here. (It would be possible, of course, to change the Authenticator's settings before every search, but then we would run into race conditions.)

[luccioman]

Ok @JeremyRand it is indeed possible the JDK itself would not be sufficient to fulfill all the needs... To answer your initial question, I can't speak for other YaCy developers, but personally I don't plan to spend time soon on this subject. If you start your own integration, don't hesitate to talk about any necessary clarifications to avoid unnecessary coding. Best regards

[linkerlin]

Try: JAVA_ARGS="-DsocksProxyHost=127.0.0.1 -DsocksProxyPort=1080 -XX:+UseCompressedOops -XX:+UseFastAccessorMethods -XX:+UseG1GC -XX:+UseStringDeduplication -server -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djsse.enableSNIExtension=false";

[JeremyRand]

Linker Lin:

Try: JAVA_ARGS="-DsocksProxyHost=127.0.0.1 -DsocksProxyPort=1080 -XX:+UseCompressedOops -XX:+UseFastAccessorMethods -XX:+UseG1GC -XX:+UseStringDeduplication -server -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djsse.enableSNIExtension=false";

That won't work for stream isolation; did you read the comments about that in this thread?

[JeremyRand]

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512

Note that SocksLib's readme states that it requires JDK 8+. My understanding is that YaCy currently works on JDK 7; is this correct? Do we want to upgrade YaCy's minimum requirement in order to get the better privacy that stream isolation provides?

-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJYhsO7AAoJELPy0WV4bWVwufMP/iQbdUu4BXmUIUnpX9NOwNlr Fyf0hdayH04iMth6vCgswl25oNayt92kxOEAhGAIjuitrR8J62emYPi+sXfafMfW 0v4z+hd8dmXzZaS5uoEXFuYqBXyGI4XdQ3KhIk3CeXxRN4IDT/vuAKLFFRARYUuo sXMqG86j1M8qczOj+i/xlIVbE9l/0RulQ2o7SPhm60qWCJ9nZjRNTi4crBlcYDJ5 E302W4BgETNMHZbmpNTnTdcH9kjeVpn557Hka4jsGVG6ll+wKtamr/djciKxK5Yp YPfl+RoT7lrLbWjjr54eUEobL63wskUc90CMdl+f/oEnAbiLdY9nlIV1r6j0i5ks a/SMH4QyEreoHTnN3z7Gk5zxPpdoXvjueqFcWrSadY/Zsmkt9v4YAoghgxYaia4k mC1o65BNwueM5y14hdrCT59NkgXljfzA21zbZWniEytPUQ6t5Xk3oepyf6eHAlCw is4q55jAkclozoLXw3MZNjG2XYasrPXELQC5+2QEhnDkIlCK3NPTLqZpWvYk1C/P 2a/mSi8CsPkO/XO9atN6trsduFgvP0DouXE+S0TEvegiZuDj4leFboxmKYWxFbE2 zs/nhQ5dnv/PpIvWqDny7aOWH7M1Lj764hnnGY8oTgPgiYMiXCaa2t/P4PqB/sp2 WCXUJe+9NtF/ri0VEW5z =KpdR -----END PGP SIGNATURE-----

[JeremyRand]

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512

I did some more research on SOCKS libraries for Java and found 2 other candidates besides SocksLib. I've added all 3 libraries to the relevant Tor wiki page:

https://trac.torproject.org/projects/tor/wiki/doc/SupportPrograms#SOCKSL ibraries

It looks like the very awesome people at Bitsquare are working on a rewritten version of their Java Tor library:

https://github.com/bitsquare/bitsquare/issues/731

So we may want to wait for Bitsquare's work on this to mature before we make decisions on what libraries to use for YaCy.

-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJYmCyMAAoJELPy0WV4bWVw2uAP/09suXCDdkk97JuZ2pN13Ti/ Eo6M2BVARbnWK3vhFRwWhrPdnPOOn8vwV3CxAqB8rLg4ltXVR45Hivmdthy6tPPA 7Hj8VQg+jy42MLcHbj5So08VLdX6KGU+lBS3mH2aPOHQvhdNpfLhlKFoaAhyzWc1 YylKHGOKrvxEqGa8mAhVZFJwTfvSPAMehSSfdgH/8n959DT9cFeNnFmPOPSPrPpW O221KKly1XHQezifC+Rzob/qJWMhMOSBFyHs+yZGpxhw1h7LrwJEeaOyzBBn20XR ABb7y1vk8pUQvvTXKpPyG1i0UQ2uNvbosSVNxR3bX6pzocZ0P4oYntIqPyudqA5Q aR75rqCNEYqnkj5BIW0hjC8jfKlkUYbuwQJ403NzEl1a6lNbsk9KjCMaudT9NuPS D/tmtn48Zs8oynbDC33eVSkgwaXfghcsiHMq7lwIUtNZUYurt17aDAbuOw2Lfr/R qv363noqe2j2L5/tJ1vJbB5Kg1JS0arBKWO7H+AzTUCiXmO3GBWNipwv4Bpbr1P6 XAcS3QLElSnfS5n1C3yq/PF6IHysM7Lpeici5mDY3MQzWBFIS3ag76wbU3MmAjI6 VH2dn9E8xut62veWD8OcALepruNY/57VEerDK/bbMu7ltUYsMUpPNH1xeO7Lcnsj hYULRIBVDZqKrqlAyezY =BaWQ -----END PGP SIGNATURE-----

[JeremyRand]

Manfred Karrer from Bitsquare informed me on Matrix that their Java Tor library should work on Java 7, possibly even Java 6.

That suggests that integrating the Bitsquare library is likely to be less disruptive to YaCy than using SocksLib.

Also, Bitsquare's SOCKS library comes with a companion library that automatically handles significant parts of Tor integration.

Given that the Bitsquare developers seem to have a pretty good track record of taking privacy (and usability) seriously, and their support for Java 7 and Tor integration, I'm definitely leaning toward using Bitsquare's library instead of SocksLib.

Opinions?

[luccioman]

Hi @JeremyRand, I am still not a Tor user, but integrating a library actively used and maintained and moreover fitting current YaCy minimum prerequisites sounds reasonable to me.

[JeremyRand]

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512

JeremyRand:

Manfred Karrer from Bitsquare informed me on Matrix that their Java Tor library should work on Java 7, possibly even Java 6.

That suggests that integrating the Bitsquare library is likely to be less disruptive to YaCy than using SocksLib.

As of 6fe735945da97abcbb91ac545fb11cff9d48effc (5 days ago) it looks like YaCy is now using Java 8, which would eliminate this particular benefit of using Bitsquare's library. I still think Bitsquare's library is preferable over SocksLib for the other cited reasons (actively used by a privacy-conscious project, and better Tor integration).

-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJZQIsrAAoJELPy0WV4bWVwVeAQAKZohxfNjVJwUCIsttYsuEYp epSUFOmrcvU1INC1YJUAbX1E8OeyDl/mlnSzh9p+/yBezxxVsrqW6K4n/4G8QLfV 4No+hMzJRWo59RONDqdKvDokABBylpWgU6JBb1r3QtVyDOwUvVOfwFJuo50j3Fjj cIWVzwBwkOaKiuJm/Ev+WBcexnMqrACjlXI/QEggrrHy2uDJe3KgbCo1wbeOolSd S6yiv8NstIt2kceHd/vJHsKjJAcPOQimUNfapVec+F9JydiC7p9e0RUs0lCaGvQg sVhFSQcngD45EWkrPCwYw7MgKn65uM9lNKK7VxHZPylf/zEq0mJxe4etkWK0P94f QNP25+O/bE5k41RhsdLqA6eBZPUGBbYrZhIWjD7iQvb8f5Y+OzK99buVSLltipAM Z8hAJBX3jP4RhWlxSaayxIwK3Q0kZqCGVnSzBqk3HQE/MEKe6xredk5MBMe57uVt /7Uba0DwrXGx7VREtMhhmDiY1WDHpIGHcimnfNK+B3Rzi1E1u++i+Irdc4abZT8s C/ubyaNsSVj3oTAjlg+vhi/eepYNEn4+dz8Y5L+1R0vdjuYeY26lKJl3LqH85H5I ZzxJ65nG09zDA9wc1pCxgd+Hy6UZHPAfwx7QgJeZwD6WXygyi47dH61lqCpfB+Rk 6jM3hEsRf9BtzqD5fD/p =4i18 -----END PGP SIGNATURE-----

[JeremyRand]

Bisq (formerly Bitsquare) has split off their Tor library into a standalone library for other projects to use, the library is now called NetLayer. NetLayer also now is Kotlin / Java 8 (but that shouldn't be a problem for YaCy since YaCy now uses Java 8).

[JeremyRand]

NetLayer is licensed under EUPLv1.1+. AFAICT that license allows relicensing under GPLv2 (as per the EUPLv1.1 text) and GPLv3 (as per the EUPLv1.2 text) when combining NetLayer with YaCy to avoid license conflicts, but it doesn't allow relicensing under GPL versions higher than v3. (The European Commission could approve a later version of GPL as part of a future EUPL version, but it is not guaranteed that this would happen.)

YaCy currently appears to be licensed as GPLv2+. So the practical effect of using NetLayer in YaCy is that, while the core YaCy code that's currently licensed under GPLv2+ would remain under that license, the combined YaCy+NetLayer code (which is what would be distributed to YaCy end users) would no longer be licensed under GPLv2+, but instead under GPLv2/3.

Is NetLayer's current license acceptable to YaCy, or would this be a dealbreaker unless NetLayer relicenses under GPLv2+?

(Obviously I'm not a copyright lawyer.)

[JeremyRand]

@luccioman Do you have an opinion on the license issue? (I asked the NetLayer dev; he prefers to keep the current license.)