Working Peer Credentials + Bob and Alice

[Neil-Jubinville]

OK so I have a nice setup where docker compose creates a custom network of nodes. I need some assistance to clarify what I am creating in terms of the working accounts / types. They seem to talk to each other.

Launching ILP Nodes

I launch my nodes with docker-compose up. They get the following IPs each time as per my config. On the host they are routable via Traefik so no localtunnel/public is needed. I use the docker IP addresses for the http_endpoint so they can connect and peer on the docker network.

ilpnode1.local - 172.25.0.3 I term this as the master node, the first in the network.
ilpnode2.local - 172.25.0.4 I term this as nodeB or peerB

At the moment the servers run. They are empty on purpose, I disabled any auto posting of accounts so I have full control of the steps.

Creating Bob and Alice

This part is simple. I designate Bob to live in the master node, and Alice to live in the nodeB.

POST bob to ilpnode1.local

{
    "ilp_address": "local.bob",
    "asset_code": "ABC",
    "asset_scale": 9,
    "http_incoming_token": "bobtoken",
    "routing_relation":"child",
    "min_balance": 0,
    "settle_threshold":100000,
    "settle_to": 0
}

POST alice to ilpnode2.local

{
    "ilp_address": "local.alice",
    "asset_code": "ABC",
    "asset_scale": 9,
    "http_incoming_token": "alicetoken",
    "routing_relation":"child",
    "min_balance": 0,
    "settle_threshold":100000,
    "settle_to": 0
}

NETWORK STATE

At this point we have two ILP nodes each with a unique account.

PEERING

To make these nodes 'aware' of each other in my infrastructure I will now post a PEER type account in EACH of the nodes.

This account is identical in credentials and tokens for simplicity with ONLY ONE CHANGE, the http_endpoint. For mutual peering the http_endpoint must point to the other respective node.

POST a peer account for nodeB in MASTER

{
    "ilp_address": "local.peerB",
    "asset_code": "ABC",
    "asset_scale": 9,
    "http_incoming_token": "peertokenB",
    "routing_relation":"peer",
    "send_routes":true,
    "receive_routes":true,
    "min_balance": 0,
    "settle_threshold":100000,
    "http_outgoing_token":"peertokenB",
    "http_endpoint": "http://172.25.0.4:7770/ilp",
    "settle_to": 0
}

POST a peer account for master in NODE B

{
    "ilp_address": "local.peer.master",
    "asset_code": "ABC",
    "asset_scale": 9,
    "http_incoming_token": "peertokenB",
    "routing_relation":"peer",
    "send_routes":true,
    "receive_routes":true,
    "min_balance": 0,
    "settle_threshold":100000,
    "http_outgoing_token":"peertokenB",
    "http_endpoint": "http://172.25.0.3:7770/ilp",
    "settle_to": 0
}

Issues and Expected Behaviour

Once these peer accounts are posted, what should happen?

I do see in the log "Sending outgoing ILP over HTTP " to the new peer entry, and on the receiving server I see "Got Incoming ILP over HTTP packet from account 1" If I trace through the messages I get to "Error sending route update" REJECT F00 Gap in routing table ...

• Should the CCP update trade details about the child accounts?
• What should I see in Redis if they successfully peered?

The fact that they each seemed to be able to decode an ILP packet tells me they are peered in authorized communication as needed because decode requires successful auth.

I also tried from my 'wallet app' to craft an ILP packet from Bob and sent it into the master node where bob has his account. The Prepare Packet had alice as the destination.

The master node replied : No route found for request { ... dump of prepare packet...}

I was expecting that the master node would look at its peers to attempt to forward the packet and find a destination almost like a default gateway. I think I can fix this by expanding my destination address to include the full URL of alice's address.

So first thing to tackle is the successful peering, please help me to confirm those payloads I am POSTING. (Is there a way to init a balance in the same call for the user accounts of bob and alice? )

Neil

[emschwartz]

"Error sending route update" REJECT F00 Gap in routing table

That's a very strange error. Can you share the full logs?

Should the CCP update trade details about the child accounts?

Every 30 seconds, the CCP route broadcaster checks the store for accounts and should broadcast the details to the peers (you might need to wait for it)

What should I see in Redis if they successfully peered?

Check out the hashmap under the key routes:current and see whether the ILP address for the other node is in there

I think I can fix this by expanding my destination address to include the full URL of alice's address.

What are you setting as the destination address in the packet? Is it local.alice? It shouldn't have anything to do with the URL of the node

Is there a way to init a balance in the same call for the user accounts of bob and alice?

Just set the min_balance to -1000000 and then you won't need to initialize the balance to be positive.

[matdehaast]

@emschwartz the Gap in routing table could be a similar issue I was experiencing when rewriting the CCP for Rafiki. What I have done is that if the epoch gotten is ahead of the expected epoch, I send a new route control message from the receiver to attempt a 'resync'. Here is the line

https://github.com/interledgerjs/rafiki/blob/d8551541b49c7a4fe1b0af90188e4d4beab31f03/src/protocols/ccp/ccp-receiver.ts#L90

[Neil-Jubinville]

Thx @matdehaast I'll look into this resync, I wonder if the adding of accounts moves the epoch index? I'll have to dig into the code more on the CCP.

@emschwartz yes the destination is simply local.alice, same error with g.alice.

The full log is here showing the gap error, note that both nodes blend their logs together, you can tell which server is outputting log data by the ilp-node-1 or ilp-node-2 :

https://gist.github.com/Neil-Jubinville/ae2352171a673109700c5a076c52f32b#file-ilp-gap-in-routing-table.

The events of interest: Line 24 - bob added to the master node. Line 29 - alice to the peer node B Line 59 - local.peerB peer added to master node Line 65 - local.peer.master add to the peer node B

Line 76 - Route updates seem to start.

Line 91 - Gap Error

[emschwartz]

Tracking this error in https://github.com/emschwartz/interledger-rs/issues/155