CardanoSolutions / ogmios

❇️ A WebSocket JSON/RPC bridge for Cardano
https://ogmios.dev
Mozilla Public License 2.0
303 stars 91 forks source link

Query UTXO by Address. #199

Closed bakon11 closed 2 years ago

bakon11 commented 2 years ago

Hi,

Im currently using the local state query to get address current UTXOs with cardano-node 1.34.1 without any issues.

However I read that this is supposed to be deprecated? If this is correct what is the recommended method to aquire address current UTXOs?

Thanks.

{
  "query": {
    "utxo": [
      "addr1wx66ue36465w2qq40005h2hadad6pnjht8mu6sgplsfj74qhpnf3s",
      "addr1xyxefct5wvh0n2h88uu44dz9q7l6nq7k2q3uzx54ruxr9e93ddt0tmqxf0n2c09tvq67lt5xkdnvc0wy5r2hzcpawrjsjk6m63"
    ]
  }
}
KtorZ commented 2 years ago

Hey, the recommended method now is to build / use a chain-index for resolving output references associated to addresses. Some 3rd-party APIs such as Blockfrost or Koios provides such functionality. If you want to remain in control of your infrastructure, I can recommended tools such as Kupo or Oura for keeping track of the chain on your behalf.

bakon11 commented 2 years ago

I could Use Oura that's for sure since I already use it. Thanks for the heads up.

Just help me understand. How does requiring you to run more and more infrastructure to do simple things like querying an address' UTXOs help with decentralization?

I feel like I'm propagating the same data twice, one with cardano-node and one with whatever solution like Oura for example, seems a little repetitive no?

Do you know, will cardano-cli still support querying UTXO's by address?

Thank you.

KtorZ commented 2 years ago

I don't see the connection between decentralization and running different software components. Having many small components, in the spirit of the UNIX philosophy, arguably makes it easier for people to build solutions using just what they need for their projects.

Also, querying UTxO by address is unfortunately not "simple". Behind the scene, UTxO are stored as an index of output references to outputs. Thus, looking up UTxOs by output references can be achieved in logarithmic (in the size of the UTxO set) complexity, but, looking up UTxO by address requires a linear scan of the set which, in the best case, is of linear complexity. As the UTxO grows and reaches millions of entries, a single lookup becomes a quite expensive operation. So long as the UTxO was held in plain "hot" memory it didn't matter too much. But with the recent updates, the UTxO is now stored on disk, in multiple shards. Thus, looking up a single entry by address requires many filesystem access and is really heavy on the CPU. It doesn't scale.

Arguably, you could say that the node could maintain a reverse index itself, and makes it easier. But the node is a semi-formally verified and critical piece of software, so one wants to minimise the inherent complexity of that software as much as possible. This offloading the tasks to external components.

The cardano-cli, like Ogmios, still supports querying UTxO by addresses so long as the node supports it. Yet, it is subjects to the same limitations and will eventually be removed all the same.

bakon11 commented 2 years ago

Ahh first off thank you for the explanation all of this makes a lot more sense now really especially the bit on how UTXOs are actually stored and the underlying changes to the node storage especially the sharding part.

I guess since I'm already using Kupo and Oura to do other tasks it's not much of a hassle to handle one extra task.

Thank you again.