synchronize peer and channel data in fiber network

Magickbase / ckb-explorer-public-issues

CKB Explorer Issues

https://explorer.nervos.org/

3 stars 2 forks source link

synchronize peer and channel data in fiber network #787

Open rabbitz opened 2 months ago

rabbitz commented 2 months ago

Create or Update Fiber Peer

POST /api/v2/fiber/peers

request

{
"name": "test", # Peer alias
"rpc_listening_addr": "http://127.0.0.1:8227",  # Peer listening address
"peer_id": "QmVCtxcWjouWHyfsjzjJCP18YDdfiYfBMMqxKwF9wmB6CT"  # Peer ID
}

response
- If the peer_id validation succeeds, it returns 204 No Content.
- If the peer_id validation fails, the response is as follows, where detail contains the error message returned by the RPC:
```
[
{
  "title": "fiber peer params invalid",
  "detail": "not support hash code at line 1 column 60",
  "code": 2010,
  "status": 404
}
]
```

Peers list

GET /api/v2/fiber/peers
params
- page # default is 1
- page_size # default is 10

response

{
  "data": {
      "fiber_peers": [
          {
              "name": "test",
              "peer_id": "QmVCtxcWjouWHyfsjzjJCP18YDdfiYfBMMqxKwF9wmB6CT",
              "rpc_listening_addr": "http://127.0.0.1:8227",
              "first_channel_opened_at": null,
              "last_channel_updated_at": null,
              "channels_count": 1,
              "total_local_balance": "193800000000.0"
          }
      ]
  },
"meta": {
        "total": 1,
        "page_size": 10
}
}

Peer detail

GET /api/v2/fiber/peers/{peer_id}

response

{
  "data": {
      "peer_id": "QmVCtxcWjouWHyfsjzjJCP18YDdfiYfBMMqxKwF9wmB6CT",
      "rpc_listening_addr": "http://127.0.0.1:8227",
      "first_channel_opened_at": null,
      "last_channel_updated_at": null,
      "fiber_channels": [
        {
            "peer_id": "QmdYroJM6VAmWGqRHa2GCEvxBTUE7Q4eW4gcm38M8foYxy",
            "channel_id": "0xf6c9802ef1baeef8f18c0c15887652e76d97244388771c178e2929cee11c4e41",
            "state_name": "CHANNEL_READY",
            "state_flags": []
        },
        ...
    ]
  }
}

Channel detail

GET /api/v2/fiber/channels/{channel_id}

response

{
  "data": {
      "channel_id": "0xf74c5362e7b5639a936b67acd1e761b4693a063f3956e35bfc0875aabcf6e96f",
      "state_name": "CHANNEL_READY",
      "state_flags": [],
      "shutdown_at": null,
      "created_at": "2024-09-12T09:22:42.000+08:00",
      "updated_at": "2024-09-19T20:41:23.111+08:00",
      "local_balance": "193800000000.0",
      "offered_tlc_balance": "0.0",
      "remote_balance": "0.0",
      "received_tlc_balance": "0.0"
  }
}

Keith-CY commented 2 months ago

Need support from the Fiber Network team to add RPCs for retrieving peer list and channel-open/close transaction list.

RPCs are expected to be added in 2 weeks.

Keith-CY commented 2 months ago

Frontend PR: https://github.com/Magickbase/ckb-explorer-frontend/pull/426

rabbitz commented 1 month ago

Graph Nodes

GET /api/v2/fiber/graph_nodes
params
- page # default is 1
- page_size # default is 10

response

{
"data": {
"fiber_graph_nodes": [
  {
    "alias": "",
    "node_id": "024714ca19abea4ddc0f3863ffdfb2e2cee76af87c477de4bc67c74a83f8140042",
    "addresses": [
      "/ip4/0.0.0.0/tcp/8228/p2p/Qmes1EBD4yNo9Ywkfe6eRw9tG1nVNGLDmMud1xJMsoYFKy",
      "/ip4/54.179.226.154/tcp/8228/p2p/Qmes1EBD4yNo9Ywkfe6eRw9tG1nVNGLDmMud1xJMsoYFKy/p2p/Qmes1EBD4yNo9Ywkfe6eRw9tG1nVNGLDmMud1xJMsoYFKy"
    ],
    "timestamp": "1728541089425",
    "chain_hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "auto_accept_min_ckb_funding_amount": ""
  },
  ...
]
},
"meta": {
"total": 4,
"page_size": 10
}
}

Graph Node Detail

GET /api/v2/fiber/graph_nodes/{node_id}

response

{
"data": {
"alias": "",
"node_id": "026ff293a8e0990dc73108e77a7cc09c5f7069b743a11d49b6a7885e03657912a7",
"addresses": [
  "/ip4/127.0.0.1/tcp/8228/p2p/QmRX6L8LiHSGLyqu6zArXx3FLTtL8KuEdqyB1e1KeqMrrd"
],
"timestamp": "1727683693959",
"chain_hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
"auto_accept_min_ckb_funding_amount": "",
"fiber_graph_channels": [
  {
    "channel_outpoint": "0xe29000aff6950883beca7e6c8329dc99afd5044c25547f11959e70f50ca0a06800000000",
    "node1": "026ff293a8e0990dc73108e77a7cc09c5f7069b743a11d49b6a7885e03657912a7",
    "node2": "02849f365a83aea567de017075402d78fadbce6af6828fbaf6b0a9be1cbc76e460",
    "chain_hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "funding_tx_block_number": "14746323",
    "funding_tx_index": "0",
    "last_updated_timestamp": "1727683757684",
    "created_timestamp": "1727683757683",
    "node1_to_node2_fee_rate": "1000",
    "node2_to_node1_fee_rate": "1000",
    "capacity": "206200000000.0"
  }
]
}
}

Graph Channels

GET /api/v2/fiber/graph_channels
params
- page # default is 1
- page_size # default is 10

response

{
"data": {
    "fiber_graph_channels": [
        {
            "channel_outpoint": "0xe29000aff6950883beca7e6c8329dc99afd5044c25547f11959e70f50ca0a06800000000",
            "node1": "026ff293a8e0990dc73108e77a7cc09c5f7069b743a11d49b6a7885e03657912a7",
            "node2": "02849f365a83aea567de017075402d78fadbce6af6828fbaf6b0a9be1cbc76e460",
            "chain_hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
            "funding_tx_block_number": "14746323",
            "funding_tx_index": "0",
            "last_updated_timestamp": "1727683757684",
            "created_timestamp": "1727683757683",
            "node1_to_node2_fee_rate": "1000",
            "node2_to_node1_fee_rate": "1000",
            "capacity": "206200000000.0"
        }
    ]
},
"meta": {
    "total": 1,
    "page_size": 10
}
}

Keith-CY commented 1 month ago

Now we have basic data from graph_nodes and graph_channels RPC(https://github.com/nervosnetwork/fiber/tree/main/src/rpc#graph_nodes), following aggregated data can be supported further:

total capacity of a graph node: summarize capacity of channels bound to a graph node
open/close transaction of a graph channel: each graph channel has its own channel_outpoint, where the funding is. The transaction generates the output is an open transaction, while the transaction consumes it is a close one
tokens in a graph channel: udt_cfg_infos is returned in graph node, the token info can be retrieved from the explorer's database according to the type script
token in a graph channel: udt_type_script is returned in graph channel, and the token info can be retrieved from the explorer's database according to the type script
nodes connected to a graph node: remote nodes connected to a graph node can be aggregated from its graph channels, for now, only the node id/pub key is necessary.

cc @rabbitz

rabbitz commented 1 month ago

Graph Node Enhancements:
- add total_capacity to the node list and details for capacity stats
- include udt_cfg_infos in the node list and details to show UDT info like full_name, symbol, decimal, and icon_file
- add connected_node_ids to show connected node IDs
Graph Channel Enhancements:
- add open_transaction_hash for the opening transaction
- include udt_cfg_info in the channel details to show UDT information

Keith-CY commented 1 month ago

Graph Node Enhancements:

add total_capacity to the node list and details for capacity stats

include udt_cfg_infos in the node list and details to show UDT info like full_name, symbol, decimal, and icon_file

add connected_node_ids to show connected node IDs

Graph Channel Enhancements:

add open_transaction_hash for the opening transaction

include udt_cfg_info in the channel details to show UDT information

total capacity, udt info and connected nodes have been added.

I found that open transaction of a channel can be located, while a channel belongs to a node, so we can have a open transaction list of a node

It should cover open transaction of all open/closed channels, and can be sorted by time. Following fields of a transaction are required:

tx hash
block number
timestamp
funding amount(ckb and udt)
funding address
transaction fee

Speaking of channels of a node, I'm not sure if closed channels are listed in the response. I hope we can get the list of closed channel, either by the current API or a new API.

For the graph nodes API, I hope a new field that count of open channels can be added

To display the share of each node, e.g. capacity/udt_amount of node / total capacity/udt_amount in fiber network, I hope there could be an API of amount of various assets, e.g.

{
  "capacity": "100000",
  "udts": [
    {
      "type_hash": "type hash of RUST",
      "decimal": "8",
      "amount": "1000000",
      "symbol": "RUST"
    }
  ]
}

@rabbitz

rabbitz commented 1 month ago

Graph Node Enhancements:

add open_channels_count to the node list

in node details, replace open_transaction_hash with outpoint_info in channel data:

"outpoint_info": {
"tx_hash": "0xb5044450a8f93f3b78b4dda13eb4a60210107c67e6de780f990d6e1cae1c953c",
"block_number": 14983532,
"block_timestamp": 1729582163892,
"transaction_fee": 920,
"funding_capacity": "106200000000.0",
"funding_udt_amount": null,
"funding_address": "ckt1qpkx0zrlugq7urrc20ckstqtwlqwvg2qgnq4d364sf5njz52lfkhcqfc6qhc4l5luvc5k0ma2ts8f826mjyqqhc0p2l7w"
}

add status param filter to node details (open by default; returns closed channels when set to closed)

Graph Channel Enhancements:
- add outpoint_info to the channel list
every 5 minutes, check open_channels; if the funding_cell is consumed, mark the channel as closed.

Keith-CY commented 4 days ago

Feature request from Cell Studio

Support searching fiber: channel id, node id, node name, pubkey, rpc address

cc @rabbitz

rabbitz commented 1 day ago

Feature request from Cell Studio

Support searching fiber: channel id, node id, node name, pubkey, rpc address

cc @rabbitz

Added filter parameter q to the graph node list, supporting exact matches on alias, peer_id, and node_id.
/api/v1/suggest_queries now supports graph node quires.

Issues:

graph channel does not return the channel_id, so filtering is not supported.
node_id is the public key; filter by node_id is now supported.

https://github.com/rabbitz/ckb-explorer/commit/76d5f195b22677ce38a3748899136ba7c5a88152