search

RGB-Tools / rgb-lightning-node

MIT License
17 stars 19 forks source link

Broking RLN API on /sendpayment though routing node with no in/out liquidity #32

Closed gofman8 closed 1 month ago

gofman8 commented 2 months ago

During our work on RGB OS/Thunderstack Cloud we faced an issue with RLN API critical error. Below we're attaching exact steps to reproduce the issue. Next steps expects to have up and running and initialized a 3 nodes: A,B,C.

Send BTC to Nodes

Run the following command on your server:

./regtest.sh sendtoaddress <address> <amount>

Example:

./regtest.sh sendtoaddress bcrt1qnc5y6j6dmejrkwy93farhvpezk0lf46gk7aecs 10

then

./regtest.sh mine 10

do it for A, B, C

Node B:

1. Issue TUSD asset (200 tokens) (RGB on-chain API)

1.1 Create Asset

Call the /createutxos endpoint:

{
  "up_to": false,
  "num": 4
}

Call the /issueassetnia endpoint with the following example body:

{
  "amounts": [200],
  "ticker": "USDT",
  "name": "Tether",
  "precision": 0
}

rgb:2RCNGGK-SPo4jJYWx-uBR5XwQd4-mUAfzyten-gzxMLo1vG-7Ktxsxp

/assetbalance node B

 {
  "settled": 200,
  "future": 200,
  "spendable": 200,
  "offchain_outbound": 0,
  "offchain_inbound": 0
}

2. Open channel with Node A with 50 tokens (Channels API)

/assetbalance balance Node B

 {
  "settled": 200,
  "future": 150,
  "spendable": 0,
  "offchain_outbound": 50,
  "offchain_inbound": 0
}

after mine 

    {
      "settled": 150,
      "future": 150,
      "spendable": 150,
      "offchain_outbound": 50,
      "offchain_inbound": 0
    }

/assetbalance balance Node A

    {
      "settled": 0,
      "future": 0,
      "spendable": 0,
      "offchain_outbound": 0,
      "offchain_inbound": 50
    }

3. Open channel with Node C with 50 tokens (Channels API)

/assetbalance balance Node B

{
  "settled": 150,
  "future": 150,
  "spendable": 150,
  "offchain_outbound": 100,
  "offchain_inbound": 0
}

after mine 

{
  "settled": 100,
  "future": 100,
  "spendable": 100,
  "offchain_outbound": 100,
  "offchain_inbound": 0
}

/assetbalance balance Node C

{
  "settled": 0,
  "future": 0,
  "spendable": 0,
  "offchain_outbound": 0,
  "offchain_inbound": 50
}

4. Write down inbound/outbound liquidity state for A,B,C here:

Action A set A fut A spen A out A in B set B fut B spen B out B in C set C fut C spen C out C in
1. Issue TUSD asset (200 tokens) 0 0 0 0 0 200 200 200 0 0 0 0 0 0 0
2. Check asset balance after issuing TUSD 0 0 0 0 0 200 200 200 0 0 0 0 0 0 0
3. Open channel with Node A (50 tokens) 0 0 0 0 50 200 150 0 50 0 0 0 0 0 0
4. After mining 0 0 0 0 50 150 150 150 50 0 0 0 0 0 0
5. Open channel with Node C (50 tokens) 0 0 0 0 50 150 150 150 100 0 0 0 0 0 50
6. After mining 0 0 0 0 50 100 100 100 100 0 0 0 0 0 50

5. Send 50 tokens to Node A (Payment API)

call /keysend on node B and sent 50 assets to Node A

 {
  "dest_pubkey": "02498860e5a6bb3bf740e1789116e9444bc514b3986cbfee7120b77cafefec07e4",
  "amt_msat": 3000000,
  "asset_id": "rgb:2RCNGGK-SPo4jJYWx-uBR5XwQd4-mUAfzyten-gzxMLo1vG-7Ktxsxp",
  "asset_amount": 50
}

call /assetbalance for node B

{
  "settled": 100,
  "future": 100,
  "spendable": 100,
  "offchain_outbound": 50,
  "offchain_inbound": 50
}

call /assetbalance for node A 

{
  "settled": 0,
  "future": 0,
  "spendable": 0,
  "offchain_outbound": 50,
  "offchain_inbound": 0
}

6 Send Send 50 tokens to Node C (/keysend)

Node B

{
  "settled": 100,
  "future": 100,
  "spendable": 100,
  "offchain_outbound": 0,
  "offchain_inbound": 100
}

Node C

{
  "settled": 0,
  "future": 0,
  "spendable": 0,
  "offchain_outbound": 50,
  "offchain_inbound": 0
}

7. Write down inbound/outbound liquidity state for A,B,C here:

Action A set A fut A spen A out A in B set B fut B spen B out B in C set C fut C spen C out C in
1. Issue TUSD asset (200 tokens) 0 0 0 0 0 200 200 200 0 0 0 0 0 0 0
2. Check asset balance after issuing TUSD 0 0 0 0 0 200 200 200 0 0 0 0 0 0 0
3. Open channel with Node A (50 tokens) 0 0 0 0 50 200 150 0 50 0 0 0 0 0 0
4. After mining 0 0 0 0 50 150 150 150 50 0 0 0 0 0 0
5. Open channel with Node C (50 tokens) 0 0 0 0 50 150 150 150 100 0 0 0 0 0 50
6. After mining 0 0 0 0 50 100 100 100 100 0 0 0 0 0 50
7. Send 50 tokens to Node A (/keysend) 0 0 0 50 0 100 100 100 50 50 0 0 0 0 50
8. Send 50 tokens to Node C (/keysend) 0 0 0 50 0 100 100 100 0 100 0 0 0 50 0

Payment Flow Test: Node A to Node C

8.1 Try to do /sendpayment from A to C (should fail)

Node C calls /lninvoice

Request:

{
  "amt_msat": 3000000,
  "expiry_sec": 620,
  "asset_id": "rgb:2RCNGGK-SPo4jJYWx-uBR5XwQd4-mUAfzyten-gzxMLo1vG-7Ktxsxp",
  "asset_amount": 25
}

Response:

{
  "invoice": "lnbcrt30u1pngczqhdqud3jxktt5w46x7unfv9kz6mn0v3jsnp4q2tw7akr7jwqatjv5rs0nvn6lqgq0ggkn4354nq3r2gw2vr8ms84gpp5lyx53922gn97qxqlqq8cxmptfutf32nqlkvx3kqhtxpsmj0stumssp5q5hg5346ks4kljf0xry3v99y9fyn0jc8hk6fg9yw47nl88kg9ejs9qyysgqcqpcxqznvlzlwfnkyw3j2fp5u368fvk4x5r0x34y5k2h0qkh2sjjx4v8w5tyxskk642pvea8jar9dckkw7ncf4xx7vtkguknwjm50pehsuq7qpe70kkw9vd36emvxd6hakezp3wpjxktmcaafcl3t74vs0qtgrmapa9eqlrfvl5vuyxflxwuzs44q4yfuv3r4a2qmn39h6t6ch52v2nm7spfz9kze"
}

Node A decodes the invoice

Request:

{
  "amt_msat": 3000000,
  "expiry_sec": 620,
  "timestamp": 1720453143,
  "asset_id": "rgb:2RCNGGK-SPo4jJYWx-uBR5XwQd4-mUAfzyten-gzxMLo1vG-7Ktxsxp",
  "asset_amount": 25,
  "payment_hash": "f90d48954a44cbe0181f000f836c2b4f1698aa60fd9868d81759830dc9f05f37",
  "payment_secret": "052e8a46bab42b6fc92f30c91614a42a4937cb07bdb494148eafa7f39ec82e65",
  "payee_pubkey": "0296ef76c3f49c0eae4ca0e0f9b27af81007a1169d634acc111a90e53067dc0f54",
  "network": "Regtest"
}

Node A sends the payment (/sendpayment)

Request:

{
  "invoice": "lnbcrt30u1pngczqhdqud3jxktt5w46x7unfv9kz6mn0v3jsnp4q2tw7akr7jwqatjv5rs0nvn6lqgq0ggkn4354nq3r2gw2vr8ms84gpp5lyx53922gn97qxqlqq8cxmptfutf32nqlkvx3kqhtxpsmj0stumssp5q5hg5346ks4kljf0xry3v99y9fyn0jc8hk6fg9yw47nl88kg9ejs9qyysgqcqpcxqznvlzlwfnkyw3j2fp5u368fvk4x5r0x34y5k2h0qkh2sjjx4v8w5tyxskk642pvea8jar9dckkw7ncf4xx7vtkguknwjm50pehsuq7qpe70kkw9vd36emvxd6hakezp3wpjxktmcaafcl3t74vs0qtgrmapa9eqlrfvl5vuyxflxwuzs44q4yfuv3r4a2qmn39h6t6ch52v2nm7spfz9kze"
}

Response:

{
  "payment_hash": "f90d48954a44cbe0181f000f836c2b4f1698aa60fd9868d81759830dc9f05f37",
  "payment_secret": "052e8a46bab42b6fc92f30c91614a42a4937cb07bdb494148eafa7f39ec82e65",
  "status": "Pending"
}

Node B API Issues

After the above operations, some APIs on Node B stop working. For Nodes A and C, the following endpoints work:

Endpoint: /listchannels

Endpoint: /listassets and /assetbalance

Node logs 

2024-07-08T15:51:29.092327Z  INFO request{method=POST uri=/assetbalance version=HTTP/1.1}: sqlx::query: summary="SELECT \"txo\".\"idx\", \"txo\".\"txid\", \"txo\".\"vout\", …" db.statement="\n\nSELECT\n  \"txo\".\"idx\",\n  \"txo\".\"txid\",\n  \"txo\".\"vout\",\n  \"txo\".\"btc_amount\",\n  \"txo\".\"spent\",\n  \"txo\".\"exists\"\nFROM\n  \"txo\"\n" rows_affected=0 rows_returned=27 elapsed=318.154µs elapsed_secs=0.000318154
thread 'tokio-runtime-worker' panicked at /rust-lightning/lightning/src/ln/channelmanager.rs:2518:67:
called `Result::unwrap()` on an `Err` value: PoisonError { .. }
zoedberg commented 1 month ago

Thanks for finding and reporting this issue. We addressed this in https://github.com/RGB-Tools/rust-lightning/commit/30b9e301e0621b05abcd26af9d87bc36279c165a and https://github.com/RGB-Tools/rgb-lightning-node/commit/c309fbaecd891485b45b5c829ad12788ef269564.

Note that /sendpayment will not fail, since the API doesn't wait for the payment to be completed. The API will create a new payment with status Pending and after discovering there are no available routes the payment status will become Failed.

Please let us know if the fix works for you.

gofman8 commented 1 month ago

Thanks for finding and reporting this issue. We addressed this in RGB-Tools/rust-lightning@30b9e30 and c309fba.

Note that /sendpayment will not fail, since the API doesn't wait for the payment to be completed. The API will create a new payment with status Pending and after discovering there are no available routes the payment status will become Failed.

Please let us know if the fix works for you.

Many thanks for rapid response and fast fix! We just verified that bug was fully resolved and now this payment scenario works as expected!