Setting up with BCH

[phewitt]

I literally have looked everywhere on how to setup insight-api with bitcoin cash. I see that you guys have a cash branch on a lot of your repos, however I have no idea how these tie into getting them to work with insight-api. I also have seem many other people asking for this. Is there ANYWHERE I can find some guidance on how to at least begin setting up BCH to work with insight.

[maektwain]

Its quite easy

1. Start your node

2. Configure bitcore with Bitcioin cash by adding this inside you config so, my config looks like this

{
  "version": "5.0.0-beta.44",
  "network": "livenet",
  "port": 3001,
  "services": [
    "address",
    "block",
    "db",
    "fee",
    "header",
    "mempool",
    "p2p",
    "timestamp",
    "transaction",
    "web",
    "insight-api",
    "insight-ui"
  ],
  "datadir": "/home/username/.bitcore/data",
  "servicesConfig": {
    "insight-api": {
      "cwdRequirePath": "insight-api",
      "routePrefix": "api"
    },
    "insight-ui": {
      "cwdRequirePath": "insight-ui",
      "routePrefix": "",
      "apiPrefix": "api"
    },
    "p2p": {
      "peers": [
        {
          "ip": {
            "v4": "nodeIP"
          },
          "port": "8333"
        }
      ]
    }
  }
}

But there is a problem with bitcore-lib which come pre-installed(the network properties belongs to bitcoin) , either you can change that before installing bitcore by switching to cash branch.(to be clear bitcore-lib is this https://github.com/bitpay/bitcore-lib/blob/master/lib/networks.js)

Now once everything is done.

Fire up.

It will work

[dviramontes]

@maektwain what version of insight-api are you using with that config ^^^ ?

[j4ys0n]

Been trying to get this working for a couple days now - just tried the strategy above, no luck.

it runs and connects to the bitcoin-abc node but i get "Invalid address" when i try to request data for a bitcoin cash address. Is there something else that needs to happen?

[j4ys0n]

@phewitt @dviramontes did either of you get this working?

[maektwain]

@dviramontes I am using the latest clone from the git.

1. Invalid address is because the conversion strategy is old, you need to manually update the conversion strategy from legacy to bitcoin cash.

This is how I did it.

1. nano node_modules/insight-api/lib/addresstranslator.js

Remove your existing AddressTranslator.translate function to this

AddressTranslator.translate = function(addresses, coin, origCoin) {
  var wasArray = true;
    console.log(coin + "Coin at coin bch");
    console.log(origCoin + "Coin at origCoin btc");
  if (!_.isArray(addresses)) {
    wasArray = false;
    addresses = [addresses];
  }
origCoin = origCoin || AddressTranslator.getAddressCoin(addresses[0]);
console.log("getAddresCoin Method" + origCoin);
  var ret =  _.map(addresses, function(x) {
    console.log(x);
    var orig = new Bitcore_[origCoin].Address(x).toObject();
if (origCoin === 'bch') {
      return Bitcore_[coin].Address.fromObject(orig).toString();
    }
var origC = Bitcore_[coin].Address.fromObject(orig).toString();
var bchAddress = Bitcore_['bch'].Address.fromObject(orig).toString();
console.log("BCH Address" + bchAddress);
console.log("Inside the map " + origC);
    return Bitcore_[coin].Address.fromObject(orig).toCashAddress();
  });

  if (wasArray) 
    return ret;
  else 
    return ret[0];

};

[buraktt]

You can add usecashaddr=0 to bitcoin.conf to use legacy addresses.

[zhousi666]

@maektwain can you tell me how to Setting up with bitcoin cash node , i can't find node_modules/insight-api/lib/addresstranslator.js The following is my deployment process： (1) install bitcore cd /root npm i npm@latest -g npm install -g --unsafe-perm bitcore@4.1.1 bitcore create bitcore-cash-node (2) install insight-api insight-ui /root/bitcore-cash-node bitcore install insight-api insight-ui (3) link bitcoin-abc.(bitcoind) /usr/local/lib/node_modules/bitcore/node_modules/bitcore-node/bin https://download.bitcoinabc.org/0.17.2/linux/bitcoin-abc-0.17.2-x86_64-linux-gnu.tar.gz ./ tar -xzf bitcoin-abc-0.17.2-x86_64-linux-gnu.tar.gz ln -sf bitcoin-abc-0.17.2/bin/bitcoind bitcoind

[zjb0807]

@maektwain "But there is a problem with bitcore-lib which come pre-installed(the network properties belongs to bitcoin) , either you can change that before installing bitcore by switching to cash branch" how to do this? thanks!

[phewitt]

I was able to get it to work with bitcore-node@3.1.3 and adding usecashaddr=0 to my bitcoin.conf. I used this bitcoin cash node https://api.github.com/repos/bitprim/bitcoin-abc/tarball/tag0.17.1-bitcore. Hope that helps!

Basically. download and compile the bitprim bitcoin-abc node (you have to have modified versions of the underlying nodes that have added the required RPC methods for insight) npm install -g bitcore-node@3.1.3 bitcore-node create mynode bitcore-node install insight-api then either run the bitcoincash node to generate the bitcoin.conf file or copy the file where you set the data directory. add usecashaddr=0 to the bitcoin.conf (for legacy format). Make sure you have insight config pointed at the bitcoin cash node and not the bitcoin node that it installs by default. Hope this helps someone!

[joadr]

Does this branch work? https://github.com/bitpay/insight-api/tree/cash

or I have to do all the above?

[psiofxt]

@phewitt Thanks a bunch for your steps -- I pretty much did the exact same thing and everything works great except I'm having trouble with specific insight-api methods like /addr. Could you elaborate what you mean by:

Make sure you have insight config pointed at bitcoin cash node...

I can't seem to find anywhere the ability to change this config.

[joadr]

@psiofxt Could you make like a very simple step by step guide to achieve this?

Also.. How big is the blockchain size in GB once synced?

Thanks!

[psiofxt]

@joadr Absolutely. I'll type something up either tonight or tomorrow. I'm also planning on writing an even more detailed blog post/guide next week on setting up bitcore with litecoin/bitcoin/bitcoin cash all on the same server -- I'll post it here.

Currently I'm only testing with BCH testnet so it's a mere 13GB.

edit: To clarify, this is with the @latest tag install for bitcore, so 3.1.3

[phewitt]

@psiofxt if you just place it in your specified "datadir" it will not overwrite if one already exists. So, what I did is make my own place it where I specify my "datadir" directory and then fire up the node. Same goes for the bitcore-node.config just make your own place it where in the bitcore-node project directory and it should use it. Hope this helps

[phewitt]

Place your "bitcore-node.json" file in your Mynode directory or whatever you named the local node you ran bitcore-node create on. here is my config for reference.

{
  "network": "livenet",
  "port": 3001,
  "services": [
    "bitcoind",
    "insight-api",
    "web"
  ],  
  "servicesConfig": {
    "insight-api": {
      "routePrefix": "api",
      "disableRateLimiter": true
    },  
    "bitcoind": {
      "spawn": {
        "datadir": "/home/cqdev/.bitcoin",
        "exec": "/usr/local/bin/bitcoind"
      }   
    }   
  }
}

and just point datadir where you want to store chaindata and have your bitcoin.conf and point exec at the bitcoin-abc binary. I think we may have made a softlink called bitcoind instead of point directly at bitcoin-abc either way that's how I did it!

Also, I setup litecoin aswell and it is a little more involved. You can't use bitpays default repos you have to modify them. I used this fork https://github.com/litecoin-project/litecore-node. Hope it helps.

[psiofxt]

@phewitt Hmm, I believe I'm doing that except I renamed all of the Bitcoin-ABC files to bitcoinCashd for instance.

It definitely runs just fine and syncs up with the BCH testnet chain, insight is just producing strange results for anything that isn't /api/block-index/<height>. I'm using the data directory that is inside the myNode folder. My config looks identical to yours except I'm using testnet and my bitcoind spawn config is

"bitcoind": {
      "spawn": {
        "datadir": "./data",
        "exec": "/usr/local/bin/bitcoinCashd"
      }

edit: Did you change any of your bitcore = require('bitcore-lib') to require('bitcore-lib-cash') in the .js files?

[phewitt]

@psiofxt No I didn't have to change any of that because I am using the old cash address (aka bitcoin addresses) with the usecashaddr=0 in the .config. What are the results? I might be able to help you if I know your error messages or anything else.

[phewitt]

Here is my bitcoin.conf file. for the bitcoin-abc node.

whitelist=127.0.0.1
txindex=1
addressindex=1
timestampindex=1
spentindex=1
zmqpubrawtx=tcp://127.0.0.1:28332
zmqpubhashblock=tcp://127.0.0.1:28332
rpcallowip=127.0.0.1
rpcuser=bitcoin
rpcpassword=local321
uacomment=bitcore
usecashaddr=0
rpcworkqueue=32

[phewitt]

I honestly haven't used testnet either. Not sure if or why that would cause any issues. But it is worth my pointing out.

[psiofxt]

@phewitt Your help is immensely appreciated, thank you. I'm going to re-index my node and try again.

I'll get back to you with results, but it is highly likely testnet may be causing the issues. If I do /insight-api/addr/mqXbNGz9EbVTmQY3otshAbMR65RaQrsPUD (some random address on BCH testnet found on blocktrail.com/tBCC) I get a Method not found error. Examining some tx id with /insight-api/tx/<tx_hash> I see block height=-1, value=NaN and a few other odd results.

[phewitt]

@phewitt Okay so method not found issues means your underlying node doesn't have the correct RPC calls. You have to download that modified version I linked above because the normal nodes do not have all the RPC methods that Insight uses. Bitpay actually forked bitcoin and added certain RPC methods that they use. I saw this issue many times and generally means that either your insight is not pointed at the correct bitcoind / bitcoin-abc that you want OR the bitcoind /bitcoin-abc you installed doesn't actually have the additional RPC methods.

Here is a commit list of some of the additional RPC methods I have seen added to nodes. This is BTG however the same RPC methods are needed on whatever node you plan to use insight with. https://github.com/BTCGPU/BTCGPU/commit/1c1d93de551d778c7e868ebe6cff66e53d09e712

[psiofxt]

@phewitt Aha! You are a god, I didn't even notice it was a bitcore specific fork of bitcoin-abc. My bitcoin abc .tar came from https://download.bitcoinabc.org/. I'll start from scratch with this and let you know.

[phewitt]

Hope this helps others too. I wish bitpay would have made more of an effort documenting this stuff. I understand it's a lot of extra work on their end with no real gain for them though. You should look into the bitcore V8.0.0 branch too it's really neat looking and seems like it will be fixing a lot of problems with standing up other nodes other than bitcoin. https://github.com/bitpay/bitcore/tree/v8.0.0. I've been dabbling with it and it doesn't have all the same endpoints that the older api has yet, but it seems like it is more active and a MUCH better architecture IMO. Here is a high level article on some of it's advantages. https://blog.bitpay.com/bitcore-node-v8-beta/

[osagga]

@phewitt Do you know where I could find the binary for the bitcoin-abc you linked (https://api.github.com/repos/bitprim/bitcoin-abc/tarball/tag0.17.1-bitcore)? It seems like the only option is to build from source, but it would be nice if I can just use a pre-built binary (like what I can find on https://download.bitcoinabc.org/0.17.2/linux/)

[psiofxt]

@phewitt After compiling the bitcore fork of bitcoin-abc I got BCH working beautifully. Thanks again. To all those reading this from google or elsewhere I'll be writing up one-by-one steps this week to get everything working. I've glanced at v8.0.0 and it does look promising, especially being able to include multiple networks in one config file.

@osagga I looked for that option as well when I was deploying, had to build the binaries myself.

usecashaddr=0 must be included for all of this to work with bitcore version <5, but I wonder if you switch all of the .js files to bitcore-lib-cash if it would work with cash addrs.

[osagga]

@phewitt Thanks for sharing the link to the custom bitcoin-abc, the setup works!

@psiofxt I have just made a guide on how to setup bitcore with BCH according the steps in this thread, you can check it out here: https://github.com/osagga/bch-bitcore-setup/blob/master/README.md, feel free to add to it or fix a mistake.

Also I manged to modify the API to show BCH addresses (explained in the guide), so the setup should be complete now.

[hxzqlh]

@osagga The bitcore-abc((https://github.com/bitprim/bitcoin-abc/tree/tag0.17.1-bitcore) add some rpc metheod to default bitcore-abc (eg: https://download.bitcoinabc.org/0.17.1/linux/bitcoin-abc-0.17.1-x86_64-linux-gnu.tar.gz), it can work together withe insight-api(v0.4.3) that supporting /addr api. Your bitcore-node-cash use bitcore-abc(https://download.bitcoinabc.org/0.17.2/) and insight-api ("osagga/insight-api#cash_v1") that also work. Does your version insight-api add some rpc implementation(eg:/addr method)? I think the cash address supporting comes from bitcore-abc itself or insight-api.

[osagga]

@hxzqlh So for bitcore-node-cash, I didn't add any additional RPC commands, but instead, I modified the bitcore-node (v5) to handle new BCH addresses (mainly using bitcore-lib-cash instead of bitcore-lib, you can check the commits for more info). Recently I've found that the setup using bitcore-node (v3) and only modifiying the insight-api is more stable and the sync speed is way faster since the bitcore-node doesn't need to validate the blocks from the node (it's considered trusted), not like the design of v5 where the peer is not trusted, and therefore the sync speed is way slower given the verification process.

Both options work so far, it's just a design decision to chose which version. Hopefully v8 could support both options.

[hxzqlh]

@osagga I understand, as for now, there are two ways to support cash addr for bitcore-node:

1. your way: bitcore-node (v5)+insight-api(osagga/insight-api#cash_v1)+bitcore-abc-0.17.2(https://download.bitcoinabc.org/0.17.2/), sync speed is slower than follow.
2. @phewitt way: bitcore-node (v3.1.3)+insight-api(v0.4.3)+bitcore-abc-0.17.1(https://github.com/bitprim/bitcoin-abc/tree/tag0.17.1-bitcore)

as you said, you are managing to work on another insight-api(osagga/insight-api#cash_v4) that will enable input addr format as BTC lagecy address or new BCH address, thank you.

[osagga]

@hxzqlh The second way doesn't support BCH addresses by default (using insight-api v0.4.3), it can only support BTC legacy addresses. use osagga/insight-api#cash_v4 as the api if you want to enable the cash addresses

osagga/insight-api#cash_v4 is ready to use, it allows you to use BCH addresses with the second setup (bitcore-node v3.1.3) I tested it on all three networks, let me know if you gave it a try.

[haydencarlson]

@osagga Thanks I'm using your insight-api#cash_v4. Got my node syncing now, I noticed that when I try to make a request to [insight_host_url]/api/addr/bitcoincash:qzq2myx5qdvplgalgcyx4ydjm82pyhdkcyga74fzrw for example I'm getting an Invalid address. Code:-5

My bitcoin.conf

server=1
whitelist=127.0.0.1
txindex=1
addressindex=1
timestampindex=1
spentindex=1
zmqpubrawtx=tcp://127.0.0.1:28332
zmqpubhashblock=tcp://127.0.0.1:28332
rpcallowip=127.0.0.1
rpcuser=bitcoin
rpcpassword=local321
uacomment=bitcore
usecashaddr=0

Then when trying to use the same url to look up the legacy address equivalent I get Invalid address: Address has mismatched network type.. Code:1

Any suggestions?

[osagga]

@haydencarlson that's weird, can you post your bitcore-node.json ? Also it would be better if you can open an issue on my fork of insight-api about this problem.

[haydencarlson]

@osagga Ok sounds good I'll open this there.

Edit: Looks like I'm unable to open an issue there, it's not enabled.

{
  "network": "livenet",
  "port": 3024,
  "services": [
    "bitcoind",
    "insight-ui",
    "insight-api",
    "web"
  ],
  "servicesConfig": {
    "insight-ui": {
      "routePrefix": "",
      "apiPrefix": "api"
    },
    "insight-api": {
      "routePrefix": "api",
      "disableRateLimiter": true
    },
    "bitcoind": {
      "spawn": {
        "datadir": "/home/[user]/.bitcoin",
        "exec": "/usr/local/bin/bitcoind"
      }
    }
  }
}

If you re-enable it I'll repost to your fork.

[osagga]

@haydencarlson issues should be open now

[qazxcvio]

@phewitt Aha! You are a god, I didn't even notice it was a bitcore specific fork of bitcoin-abc. My bitcoin abc .tar came from https://download.bitcoinabc.org/. I'll start from scratch with this and let you know.

I also encountered the problem that bch insight /addr could not access. Return me Method not found. Code:-32601, can you ask how this is solved?

[1scrooge]

Hi All. I have the Bitcoin node with insight-api. And I have a problem with 556034 block, Insight-api can not download 556034 block, because it is large (32 Mb). Could anybody help me? Insight-api log:

[2018-11-12T06:59:11.627Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:16.632Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:21.638Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:26.643Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:31.648Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:36.650Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:41.655Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:46.660Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:51.665Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T06:59:56.670Z] info: Block Service: download progress: 556033/556297  (99.9525%)
[2018-11-12T07:00:01.676Z] info: Block Service: download progress: 556033/556297  (99.9525%)

bitcore-node.json:

{
  "version": "5.0.0-beta.44",
  "network": "livenet",
  "port": 3001,
  "services": [
    "address",
    "block",
    "db",
    "fee",
    "header",
    "mempool",
    "p2p",
    "timestamp",
    "transaction",
    "web",
    "api"
  ],
  "datadir": "/ext/bitcore",
  "servicesConfig": {
"p2p": {
      "peers": [
        {
          "ip": {
            "v4": "127.0.0.1"
          },
          "port": 8333
        }
      ]
    }

[matiu]

Hi,

Bitcore v8 was able to handle 32mb blocks without any problem.

The older version had a hard time handling them. You need to up the mem to 7.5GB and wait a long time (around 1hr) to process the 32GB on a regular server hardware.

If possible, migrate to v8 to work for BCH.

[some-dev]

What do you mean by "migrate to v8 to work for BCH." ? Could you explain a bit, please.

[some-dev]

We up the memory to 14 gb, but the issue still remain..

[msaikat]

@phewitt Thanks for sharing the link to the custom bitcoin-abc, the setup works!

@psiofxt I have just made a guide on how to setup bitcore with BCH according the steps in this thread, you can check it out here: https://github.com/osagga/bch-bitcore-setup/blob/master/README.md, feel free to add to it or fix a mistake.

Also I manged to modify the API to show BCH addresses (explained in the guide), so the setup should be complete now.

Hi @osagga,

Is there anything that can be followed for bitcoin sv. bch-bitcore-setup guide was helpful, I was looking for something similar for bsv. I could already run the test node. However, I a stuck with insight-api integration so that I can explore programatically.

Any kind of help is highly appreciated.

Thanks