Tor onion integration

[david415]

chatting with Juan Bennet at c-base, Berlin about Tor onion service integration

here's what we've identified as necessary for proper Tor integration:

1. adding /onion to go-multiaddr - /onion/<onion-key>/ipfs/<ipfs-key>
2. adding /onion dialing to go-multiaddr-net
3. make a build of go-ipfs that:
   • only uses /onion
   • only bootstrap to onion nodes
   • disable mdns service
   • disable NAT service

I know plenty of people who'd be willing to run some Tor onion bootstrap nodes.

(edited by @jbenet for links)

[lidel]

@Stebalien just for the record, would go-ipfs accept a PR to add Tor transport as opt-in experiment (behind a flag) to go-ipfs? Or would it be parked until audit happens?

[Stebalien]

I believe the best solution here is to:

1. Add support for loading transports via plugins.
2. Create a plugin for the onion transport, out of tree.

[Trumeet]

Any updates on this issue?

[xloem]

@david415 if you were willing to enumerate the reasons for your opinion, maybe people who agree with you but are closer to development decisions than we are could include them in future plans some day

(Edit: it is obviously dangerous to use ipfs for content that anyone with a lot of power or money wants to remove or track, without using tor, as you can be identified by your hosting ip address.)

[david415]

It's too late for this discussion. IPFS has failed to embrace the concept of free Tor integration from volunteer developers.

That having been said, anonymity is a synonym for traffic analysis resistance; that is to say, even encrypted traffic can be analyzed for the metadata it leaks. Tor is the very weakest of the existing designs for anonymous communication networks however it is the most widely used whereas the other designs from academia have not had much field testing; such as: mix networks, dcnets, verified shuffles and other things can be used to form anonymous communication networks such as private information retrieval, oblivious ram, multi party computation etc.

Tor is trivially broken by any sufficient global adversary by means of timing correlation whereas mixnets are not. There are many other ways to break Tor.

Anonymity aka traffic analysis resistance is not yet a popular security feature because these designs are in some respects ahead of their time... just like not every software project embraces deterministic builds. Just because your white middle class platitude doesn't allow you to understand why people in high risk situations might need these things doesn't mean they are not needed. In fact, in dealing with such folks I find the easiest way to impart the importance to them is to describe military scenarios, e.g. if you were in the military, overseas, you might actually be interested in traffic analysis resistance.

Think about a future brighter than Tor! Think about mixnets, hybrid networks, dcnets and so on. Monoculture is death. Why is Tor the only successful anonymity network? And to a lesser degree I2p? Although the I2p observation is less valid because it's design is so similar to Tor in that it can easily be broken by timing correlation from a sufficiently global adversary.

[seanlynch]

So what you're saying is that if something can be broken by a "sufficient global adversary" it's not worth using? This seems like a silly reason not to provide protection against casual snoopers, criminals, and copyright trolls.

You are welcome to pur your own efforts into a mixnet or whatever. But please don't go telling others they can't have any protection at all just because their requirements don't match yours.

On Fri, Sep 4, 2020, at 14:49, David Stainton wrote:

It's too late for this discussion. IPFS has failed to embrace the concept of free Tor integration from volunteer developers.

That having been said, anonymity is a synonym for traffic analysis resistance; that is to say, even encrypted traffic can be analyzed for the metadata it leaks. Tor is the very weakest of the existing designs for anonymous communication networks however it is the most widely used whereas the other designs from academia have not had much field testing; such as: mix networks, dcnets, verified shuffles and other things can be used to form anonymous communication networks such as private information retrieval, oblivious ram, multi party computation etc.

Tor is trivially broken by any sufficient global adversary by means of timing correlation whereas mixnets are not. There are many other ways to break Tor.

Anonymity aka traffic analysis resistance is not yet a popular security feature because these designs are in some respects ahead of their time... just like not every software project embraces deterministic builds. Just because your white middle class platitude doesn't allow you to understand why people in high risk situations might need these things doesn't mean they are not needed. In fact, in dealing with such folks I find the easiest way to impart the importance to them is to describe military scenarios, e.g. if you were in the military, overseas, you might actually be interested in traffic analysis resistance.

Think about a future brighter than Tor! Think about mixnets, hybrid networks, dcnets and so on. Monoculture is death. Why is Tor the only successful anonymity network? And to a lesser degree I2p? Although the I2p observation is less valid because it's design is so similar to Tor in that it can easily be broken by timing correlation from a sufficiently global adversary.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ipfs/notes/issues/37#issuecomment-687407153, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABI56RPO7RM2SGL6PHVYS3SEFOGDANCNFSM4BOOQVOA.

-- https://www.literati.org/~seanl/ This is a temporary email address I use for outbound emails. I will be changing it as it starts to receive spam. If you'd like a permanent address to use, please ask for one.

[xloem]

It sounds like there are newer networks that would be even better to integrate into services than Tor? Are any to or near the point of usability?

[xloem]

Just to add, I suspect the reason that the state of public anonymity tools is not stronger is that the existing international powerholders, whose power could be reduced by widespread accessible anonymity, take diverse action to slow the release and hinder the effective use of the research.

The way to make things change would be for people like us to agree to work together on forging one right thing in a development community, and use tools of both interpersonal mediation and software development to bring the result to happen by force of collective determination. It might help if everyone kept themselves more anonymous, collaborated in private as well as in public, and supported people who ran into personal issues so as to resist disruption and keep the work moving forward.

[seanlynch]

I don't think there's any need to posit a conspiracy to explain this. Strong anonymity is extremely difficult to implement, and not enough people both understand and care about it to make it actually happen.

The best proposal I've seen so far is Chaum's. But there is no stronger system than Tor or I2P that's in a usable state. Personally I trust Tor more than I2P because despite its government funding it has received far more attention from security researchers, and because the compromises have tended to be either against high-value, large targets or large numbers of lower value targets. Large targets are an obvious weakness in Tor against an adversary with wide visibility into the network, and the mass compromises have relied on more traditional means, in particular attacks against the browser.

Burning 0days is costly, so unless it's going to result in the neutralization of a high-value target or a lot of lower-value targets, it seems relatively safe to rely on Tor unless you're engaging in some behavior that's going to get the attention of some well-funded adversary. And if you are, you probably shouldn't be using IPFS either, since it could have its own vulnerabilities that would make compromising Tor unnecessary.

On Sat, Sep 5, 2020, at 10:00, xloem wrote:

Just to add, I suspect the reason that the state of public anonymity tools is not stronger is that the existing powerhouses, whose power could be reduced by accessible anonymity, take diverse action to slow the release and hinder the effective use of the research. The way to make things change would be for people like us to agree to work together on forging the right thing in a development community, and use tools of both interpersonal mediation and software development to bring the result to happen by force of collective determination. It might help if everyone kept themselves more anonymous, collaborated in private as well as in public, and supported people who ran into personal issues so as to keep the work moving forward, so as to resist disruption.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ipfs/notes/issues/37#issuecomment-687636039, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABI56RGGGWSL4NVBOFIB43SEJVETANCNFSM4BOOQVOA.

-- https://www.literati.org/~seanl/ This is a temporary email address I use for outbound emails. I will be changing it as it starts to receive spam. If you'd like a permanent address to use, please ask for one.

[xloem]

I don't think there's any need to posit a conspiracy to explain this.

Let's try to speak concisely and directly, and find ways to support each other.

There are a lot of people taking action for a lot of different reasons, but I have no need to call any of it a conspiracy. My experience is that working on democracy software can become dangerous to the project if it has global publicity, and my knowledge is that acting on this can help sustain other projects. Giving warning prominently, early, and relevently, is an easy way to turn danger into just a conspiracy theory.

I'm observing that David worked very hard for Tor here and it didn't pan out.

For example, discussion of newer protocols, or discussion encouraging a return to Tor, both could take focus, energy, time, and that developer excitement that can make hours of work seem like nothing, away from a potential developer, replying to someone but possibly disagreeing. If someone were working and took the safety of collaborating among people already in agreement, they could have had more man-hours in their project.

I'm not sure anybody here is working on this at the moment, but that's not always the case. If David did most of the work on this earlier, it could give reason to support him.

But there is no stronger system than Tor or I2P that's in a usable state.

I'm always looking to find those obscure implementations nobody talks much about. One of them will be the future Tor.

It does seem to me that Tor already has a codebase and community, and forking can be a quick way to keep working through disagreements; I haven't looked into the code.

[david415]

Dear @seanlynch you are belligerent. I am in fact not saying that Tor or I2p aren't worth using. Come off your platitude for a few minutes and think about people who may be in a more high risk situation than yourself. For those people Tor isn't good enough because their adversary may well be the NSA and the FBI and GCHQ and so on. I am welcome to put forth my efforts into mix networks but not because you say so speaking from a place of belligerence. I am a fan of Tor and I do not go around telling people they can't have any protection. Your delusional caricature of myself is offensive and alarming. Check yourself before you wreck yourself.

[aschmahmann]

It's great to see people passionate about adding some level of anonymizing support to IPFS implementations. Just a reminder that discourse in the IPFS repositories is guided by the code of conduct https://github.com/ipfs/community/blob/master/code-of-conduct.md.

As for the state of things here I agree with the suggestion above https://github.com/ipfs/notes/issues/37#issuecomment-447198558 that the best way to move this forward is for some work to be done improving the plugin system (e.g. for go-ipfs https://github.com/ipfs/go-ipfs/issues/7652 and https://github.com/ipfs/go-ipfs/issues/7653) and then for someone motivated by this to make a plugin for Tor support. If this approach interests any of you then feel free to comment on https://github.com/ipfs/go-ipfs/issues/7653 to voice support for making the libp2p transports pluggable.

[Jorropo]

I'm not sure of what is the current status about this issue, but @berty we are working on a tor transport.

This is a beta and will not protect your IP !

Basicaly I need to add a dns resolving service (all dns resolution are still made through your real ip, not through tor)

It's using a lot of CGO and CI magic to builtin a tor node for a seemless experience (the CI prebuild tor in go a package so it's CGO seemless (just import it and it works, other alternatives require manualy building a make file to produce a valid go package)). Right now it's working on Linux and Android (I'm working on macos and it is in alpha (still need some ci stuff)) and IOS is planned in the "close" future (hopefully before next week). A 1.0 should be available soon (it's lacking, dns resolving, IOS, some reffinment of the transport (such as the option to use an external tor node for server or the ability to use 0/1 hop mode)). Using 0/1 hop mode it could also be a usefull replacement of circuit (basicaly instead of using libp2p's network to relay it would use the tor network (wich is way way bigger)) the problem with that is due to how tor work it would be really really hard (close to impossible) to make it work in an offline situation.

[david415]

If I recall correctly openbazaar uses our Tor transport we've written for IPFS: https://github.com/OpenBazaar/go-onion-transport

You should not be making separate DNS queries. The Tor exit node handles this. If you do this the Tor integration will be meaningless for traffic analysis resistance because the network intermediaries such as the client's ISP will be able to identify the connection destinations via the DNS queries, obviously. Just my two cents.

I'm not clear on why exactly our work on the Tor integration hasn't been acceptable but recent comments suggest a plugin system is desired and is a prerequisite. This to me seems more like a political decision and it would've been nice to know this upfront from Juan instead of years later after our efforts have largely gone unnoticed and for the most part, ignored.

Additionally the fact that IPFS and the associated commercial money making enterprises have chosen not to pay for the Tor integration themselves speaks volumes about their priorities. (we make money not privacy)

Is it against your code of conduct to be frustrated with the way our volunteer code contributions have been treated? Is it against your code of conduct to point out that there are several glaring issues that illustrate your project as a post-privacy software project? Lack of encryption for files at rest is another issue which I admit is rather off topic for this thread but offers additional support for my claim of post-privacy.

[balupton]

Really, until we have something like https://onionshare.org/ or https://www.tribler.org/ for ipfs, I wouldn't consider this issue of privacy solved, considering the ipfs dht doxes you. It seems all the big ipfs vc money is going to centralised ipfs cloud providers, and the imitations of them. Would be nice if some went to decentralised privacy. In the end, ipfs just seems a tool for big data companies to save some coin on automatic redundancy replication and deduplication, however the need to encrypt absolutely everything on ipfs for minimal privacy (still doxed by the dht though) nearly completely eliminates such benefit, making ipfs just a slower alternative to existing cloud providers.

For alternatives, Sia and Blockstack are worth keeping an eye on, and BitTorrent v2 already works in production and has most of the advantages and use cases of IPFS. IPFS marketing makes it out as if they are the only players in town.

[Jorropo]

considering the ipfs dht doxes you

@balupton yes, it's a technical limitation of the DHT, we can't do better, I2P (a completely decentralized Tor equivalent) uses DHT instead of central server for rendez-vous meeting, but shabils attacks have been demonstred working against it, I guess you can use a blockchain instead of a DHT for rendezvous, but that costy and doesn't scale well. It's just something really hard to do, DHT is scalable and undiriged / consensus free, relatively cheap but it's mutable, if you really want to avoid DHT doxing you can use DHT + central servers for rendez-vous (it's what we do @berty) so if the rendez-vous servers are DDOS or maitainer shutdown them / manipulate them, you still have DHT and reverse, you would need to control all rendez-vous servers and a part of the DHT to mute someone.

To conclude : DHT have advantages but have drawbacks like other solution (central server and blockchain), IPFS's devs thinks DHT is the best tradeoff but nothing stop you from hosting your own central solution (like we do) or write a blockchain rendezvous driver.

PS: I just noticed, the DHT is not used on berty right now because we were having problems with it, but it's gonna be bring up later.

[balupton]

Thinking about this more. What is actually needed here for adoption, is not a client that does tor/onion routing for all files, but for only the files/dirs one considers private.

For instance, an interface like ipfs desktop, onionshare, tribler, or dropbox - they are all equivalent in this proposal - which when you drag a file into, or select a file in a add file dialog, should prompt, before it is added, which privacy mode you want for the file:

1. Open + Public: the original file will be listed, shared, and readable via your ip address to the public: best for increasing availability for public domain files (eg. public goods)
2. Open + Confidential: the encrypted file will be listed, shared, and readable via your ip address to the public, however only those who you share it with will be able to decrypt it: best for copyrighted files (eg. documents and photos)
3. Private: the original file will be listed, shared, and readable via an anonymised tor IP address to the public: best for increasing availability for files you don't want to be publicly associated with (eg. private public files, eg. controversial media)
4. Private + Encrypted: the encrypted file will be listed, shared, and readable via an anonymised tor IP address to the public, however only those who you share it with will be able to decrypt it: best for files you want to share privately, only to a select audience (eg. private confidential files, eg. early stage whistleblowing)

So the transport should be specific to the file/directory's privacy setting. The UI can distinguish the modes via colours and symbols for the file's row in the file listing, similar to browser's incognito mode.

Perhaps a moniker between these can be used: private, public, open, confidential, transparent, closed. So private vs transparent for tor vs no-tor, and closed vs open for encrypted vs non-encrypted.

For the private mode, many may be suitable with a openvpn or wireguard or cloudflare-warp transport, rather than necessarily a tor transport. VPN transports could also be a source of revenue from affiliate sales.

[Kelketek]

@balupton The issue with opt-in privacy and encryption is the anonymity set is very small. It is better if privacy is opt-out. We don't want a file or user to be targeted as especially important due to the fact it's marked private. If only a small number of files are marked as private, then those will look especially delicious to those seeking compromising or otherwise personal information.

It also makes it a trap for people who are configuring data stores and either are unaware that all the files are public or else forget to toggle the correct setting, and invites for bugs and oversights that make things that were once private not so when some recent change to a script that forgets to explicitly enable privacy restores the default.

[Kelketek]

For clarification-- your idea does not preclude privacy by default, it just points out that not all files have to be encrypted. That's true, but I'd like to suggest it should be the default that they are.

[dmrobotix]

Has anyone made any progress since this discussion last year?

[ljmf00]

I'm looking forward to this. It will add tons of value in terms of anonymization of the network

[Stebalien]

No progress has been made and this is not a priority. Unfortunately, it's a large undertaking with a high risk of going poorly. To make this happen (by default), we'd need to:

1. Audit go-ipfs for any possible information leak (IP addresses, etc.)
2. Audit go-ipfs for fingerprinting vectors.

[dagpacket]

ded

[emdee-net]

Who decided that this is not a priority?

You have to solve the tor connectivity first if you want to then address corporate proxies. Has it been decided that having IPFs run in a corporate environment is not a priority? By who?

[Jorropo]

@emdee-net IPFS is an opensource project, we don't think it's a priority means we wont work on it for now.

But if you disagree just make it yourself, or pay someone to do it, send us some code and then we will take a look at it.

There is already at least two PoC you can pickup and finish (most of what there is to do is fixing IP leaks: dns, ...). (and fingerprinting but it's a later task, at the start we can just say one node = one identity)

If we want to work on other things instead, we work on other things instead.

[RangerMauve]

For what it's worth, protocol labs is actively looking to fund people to work on content routing privacy, so folks looking to add something like TOR could start there.

https://research.protocol.ai/blog/2022/new-open-problems-in-private-data-retrieval/