Add support for Tor

[ryanxcharles]

Be able to connect through Tor to the bitcoin p2p network

[tacticalchihuahua]

wouldn't this be configured via bitcoind? from the bitcoin wiki page for tor:

Run bitcoind with -proxy=127.0.0.1:9050 (or whatever your SocksPort is). bitcoind will detect that you are using a proxy on 9050 and will force the "nolisten" flag. If you are not running tor on 9050, you need to set "nolisten" manually otherwise you will listen on your public IP and possibly reveal that you are running a node.

[ryanxcharles]

Only if you're using bitcoind. If you're connecting directly to the p2p network with bitcore, you may want to go through Tor.

[tacticalchihuahua]

ah, okay, i'd be happy to work on this. this would be something added to RpcClient and/or PeerManager? (still getting familiar with this stuff)

[jgarzik]

PeerManager most likely. There is a node "socks5" client and server out there, to connect to Tor's SOCKS5 proxy.

[tacticalchihuahua]

awesome, i'll work on this this evening

[ryanxcharles]

Awesome.

[jgarzik]

And Peer and Connection classes too, obviously.

[jgarzik]

Note that bitcoin encodes Tor onion addresses in a specially formatted IPv6 address style.

[tacticalchihuahua]

i believe i was able to get this working. i had to fork socks5-client so that the actual constructor could be used as well as proxy the socket's "data" events instead of just defining the ondata function provided by ReadableStream. i have a pull request with these updates open.

anyway, using bitcore, i was able to do this pretty easily:

var Socks5Client = require('socks5-client');
var Peer         = require('./Peer');
var Connection   = require('./Connection');

var client       = new Socks5Client('127.0.0.1', 9050);
var peer         = new Peer('188.226.187.27', 8333);
var connection   = new Connection(client, peer);

client.connect(peer.port, peer.host);

connection.on('error', function(err) {
  console.log(err);
});

And the console spits out:

➜  bitcore git:(feature/tor) ✗ node socks.js 
{ '0': '[188.226.187.27:8333] Sending message version (99 bytes)' }
{ '0': '[188.226.187.27:8333] Recieved 148 bytes of data:' }
{ '0': '... f9beb4d976657273696f6e000000000064000000f6c33c487111010001000000000000...' }
{ '0': '[188.226.187.27:8333] Received message version (100 bytes)' }
{ '0': '[188.226.187.27:8333] Sending message verack (0 bytes)' }
{ '0': '[188.226.187.27:8333] Received message verack (0 bytes)' }
{ '0': '[188.226.187.27:8333] Recieved 61 bytes of data:' }
{ '0': '... f9beb4d9696e7600000000000000000025000000fbcfa6af0101000000892b302a694b...' }
{ '0': '[188.226.187.27:8333] Received message inv (37 bytes)' }
^C% 

So now I suppose that figuring out where it makes sense to add this functionality to bitcore will be the task. :smile_cat:

On a related note, does bitcore assume responsibility for peer discovery or is this left to the user? Correct me if I am wrong but if we want to add support for Tor, won't we need to expose a way to find nodes using the same DNS seeds in the bitcoin core?

[ryanxcharles]

Great job. Peer discovery happens over the p2p network. You just need one node to connect to initially. Not sure how this works over Tor.

[tacticalchihuahua]

Working with @jgarzik briefly yesterday, I got the impression that over Tor, you need to explicitly supply the first node. That's where I am getting the IP for the Peer instance in the example above - by running:

~# dig seed.bitcoin.sipa.be

And then picking out an IP to use. We should be able to do this programmatically pretty easily if you think it makes sense to do so in bitcore.

[jgarzik]

Peer discovery is pretty much absent, in current bitcore

[tacticalchihuahua]

so this is a fairly primitive example, but i put together a SeedList class we might use that just attempts to resolve a seed list from the passed uri or tries from a default list by default:

var EventEmitter = require('events').EventEmitter;
var dns          = require('dns');
var inherits     = require('util').inherits;

var SeedList = function(options) {
  this.options = options || {};
  this.sources = [
    'dnsseed.bluematt.me',
    'dnsseed.bitcoin.dashjr.org',
    'seed.bitcoin.sipa.be',
    'seed.bitcoinstats.com',
    'bitseed.xf2.org'
  ];
  this.source  = this.options.source || this.sources[0];
  this.seeds   = [];
  this.find();
};

inherits(SeedList, EventEmitter);

SeedList.prototype.find = function() {
  var self = this;
  dns.resolve(self.source, function(err, seeds) {
    if (err) {
      var index = self.sources.indexOf(self.source);
      if (index !== -1) {
        index++;
        if (!self.sources[index]) {
          return self.emit('seedsNotFound');
        }
        else {
          self.source = self.sources[index];
        }
        self.find();
      }
      return self.emit('error', err);
    }
    self.seeds = self.seeds.concat(seeds);
    self.emit('seedsFound', seeds);
  });
  return self;
};

in my sandbox, i'm using this to get create the initial peer so the example above gets updated to:

var Socks5Client = require('socks5-client');
var Peer         = require('./Peer');
var Connection   = require('./Connection');

var seedlist = new SeedList();
var client   = new Socks5Client('127.0.0.1', 9050);

seedlist.on('seedsFound', function(seeds) {
  // use the first seed in list
  var peer       = new Peer(seeds[0], 8333);
  var connection = new Connection(client, peer);

  client.connect(peer.port, peer.host);

  connection.on('error', function(err) {
    console.log(err);
  });
});

seedlist.on('seedsNotFound', function() {
  console.log('failed to find seeds :(');
});

seedlist.on('error', function(err) {
  console.log('error:', err);
});

this is working as expected, with the same result as before. you guys think something similar to this would find a home in bitcore?

[jgarzik]

Something like that. Let's go over what the seeds are attributes are, and how that might integrate into bitcore.

• The list of DNS seeds themselves is a property of each chain. There are different seeders for livenet and testnet. That list belongs in networks.js.
• A good P2P node maintains an internal database of active, dead, or not-yet-tried addresses.
• Initially, you will populate or refresh this db with DNS seeds. There will need to be a helper class or method to query all the DNS seeds, possibly in parallel because node.js is cool like that.
• It seems like PeerManager is the place for address management, so that seems like a likely home for initiating DNS lookups and "getaddr" P2P messages, and storing the results internally.

In general this is a primitive side of bitcore, and "ask how bitcoind does it" is a good default strategy.

[matiu]

One small comment: in all modules of bitcore we use https://github.com/bitpay/soop for inheritance.

You can check PeerManager.js and Connection.js, those modules uses soop and inherits EventEmitter also.

On Wed, Mar 19, 2014 at 5:02 PM, Jeff Garzik notifications@github.comwrote:

Something like that. Let's go over what the seeds are attributes are, and how that might integrate into bitcore.

• The list of DNS seeds themselves is a property of each chain. There are different seeders for livenet and testnet. That list belongs in networks.js.
• A good P2P node maintains an internal database of active, dead, or not-yet-tried addresses.
• Initially, you will populate or refresh this db with DNS seeds. There will need to be a helper class or method to query all the DNS seeds, possibly in parallel because node.js is cool like that.
• It seems like PeerManager is the place for address management, so that seems like a likely home for initiating DNS lookups and "getaddr" P2P messages, and storing the results internally.

In general this is a primitive side of bitcore, and "ask how bitcoind does it" is a good default strategy.

— Reply to this email directly or view it on GitHubhttps://github.com/bitpay/bitcore/issues/122#issuecomment-38099850 .

Matías Alejo Garcia Skype/Twitter: @ematiu Roads? Where we're going, we don't need roads!

[tacticalchihuahua]

@matiu gotcha, i will make sure i follow this convention

[tacticalchihuahua]

@jgarzik awesome, i'll start trying to chop some this stuff up and come back with some ideas, any thoughts you've got in the meantime are valued.

[tacticalchihuahua]

socks5-client maintainer merged my PR - fyi: https://github.com/mattcg/socks5-client/pull/4

[ryanxcharles]

Awesome!

[tacticalchihuahua]

A good P2P node maintains an internal database of active, dead, or not-yet-tried addresses. Initially, you will populate or refresh this db with DNS seeds.

Should we really handle the database and storing aspect of this? Isn't bitcore more low-level than that? I get the impression that this would be up to the user to handle in their implementation.

As far as the scope of this specific issue is concerned (connecting through Tor), in my mind, this should be as simple as adding an options argument to the Connection class, where the user could specify a SOCKS5 proxy host/port, and the already passed socket object would just get handed off to to socks5-client and the Connection instance's socket member would be the "proxied" socket instead of the one passed.

In regard to "getting" a Peer object to pass to the Connection class - this seems like an entirely separate feature (peer discovery), which maybe we track and discuss separately?

[jgarzik]

Yes and no.

bitcore is low level and should not force a single database or address collection system. However, providing a default implementation of an address datastore would clearly be beneficial. I would structure it like the recent TransactionBuilder addition: it is not core functionality, but we ship it because it's useful.

[jgarzik]

The first milestone is getting this working with a single, manually-specified .onion P2P node. That eliminates any need for address databases.

[tacticalchihuahua]

PR for opening a connection through Tor: https://github.com/bitpay/bitcore/pull/210