adjust query to support ordering

45930 commented 2 days ago

Fixes: #106

This PR adds sequenceNumber, as well as zkappAccountUpdateIds to the graphQL schema, which will expose enough information for clients to verify the order in which actions were processed by consensus. It also attempts to return actions in that order, and exposes the ordering spec in a unit test.

45930 commented 1 day ago

Yes, I will have to work on the format. Here's what I can produce on the current branch:

{
    "data": {
        "actions": [
            {
                "blockInfo": {
                    "stateHash": "3NKZqA81s8iHMQc4vcJxrcx6epVB7C9vBD8oV7K3EpCZzvfebCtP",
                    "timestamp": "1729695587000",
                    "height": 69,
                    "parentHash": "3NLAneCAdNdj2u6DW5ycvUAvuJZP4mMain7eLWEU6QvL9ZTmZ53i",
                    "chainStatus": "canonical",
                    "distanceFromMaxBlockHeight": 36,
                    "globalSlotSinceGenesis": 132
                },
                "actionState": {
                    "actionStateOne": "25619375377485241455325506385026113155326956491463643967090964080969742671789",
                    "actionStateTwo": "25079927036070901246064867767436987657692091363973573142121686150614948079097",
                    "actionStateThree": "25079927036070901246064867767436987657692091363973573142121686150614948079097",
                    "actionStateFour": "25079927036070901246064867767436987657692091363973573142121686150614948079097",
                    "actionStateFive": "25079927036070901246064867767436987657692091363973573142121686150614948079097"
                },
                "actionData": [
                    {
                        "data": [
                            "24650",
                            "1",
                            "63073",
                            "4611450100234868895111341341933539196411650948711574062334330094836938200670",
                            "1"
                        ],
                        "accountUpdateId": "27",
                        "transactionInfo": {
                            "status": "applied",
                            "hash": "5JvAynd7NAHEiMqCpgZD85WCzLniBv18eSEJyuvNVK8pQL8Fi5kx",
                            "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"
                        },
                        "sequenceNumber": 2,
                        "zkappAccountUpdateIds": [
                            27,
                            28
                        ],
                        "zkappEventElementIds": [
                            23
                        ]
                    },
                    {
                        "data": [
                            "24651",
                            "1",
                            "63073",
                            "4611450100234868895111341341933539196411650948711574062334330094836938200670",
                            "1"
                        ],
                        "accountUpdateId": "28",
                        "transactionInfo": {
                            "status": "applied",
                            "hash": "5JvAynd7NAHEiMqCpgZD85WCzLniBv18eSEJyuvNVK8pQL8Fi5kx",
                            "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"
                        },
                        "sequenceNumber": 2,
                        "zkappAccountUpdateIds": [
                            27,
                            28
                        ],
                        "zkappEventElementIds": [
                            24
                        ]
                    },
                    {
                        "data": [
                            "73587",
                            "1",
                            "92297",
                            "8253926607854573680301530978054957179571865656842078211022112667228198387920",
                            "1"
                        ],
                        "accountUpdateId": "25",
                        "transactionInfo": {
                            "status": "applied",
                            "hash": "5JumftuGBSY8x4yHpqvpg4gDWUnEpuZ8335Nxr5FgsXxmUsqmX8J",
                            "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"
                        },
                        "sequenceNumber": 1,
                        "zkappAccountUpdateIds": [
                            25,
                            26
                        ],
                        "zkappEventElementIds": [
                            21
                        ]
                    },
                    {
                        "data": [
                            "73588",
                            "1",
                            "92297",
                            "8253926607854573680301530978054957179571865656842078211022112667228198387920",
                            "1"
                        ],
                        "accountUpdateId": "26",
                        "transactionInfo": {
                            "status": "applied",
                            "hash": "5JumftuGBSY8x4yHpqvpg4gDWUnEpuZ8335Nxr5FgsXxmUsqmX8J",
                            "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"
                        },
                        "sequenceNumber": 1,
                        "zkappAccountUpdateIds": [
                            25,
                            26
                        ],
                        "zkappEventElementIds": [
                            22
                        ]
                    },
                    {
                        "data": [
                            "55455",
                            "1",
                            "86171",
                            "476632679643188528985438056980162363837707337312210785567256027534077014652",
                            "0"
                        ],
                        "accountUpdateId": "23",
                        "transactionInfo": {
                            "status": "applied",
                            "hash": "5Ju8GoXZWFqBKTAv3b1iMbc77yUhZEoVc5RwD5aj3VQw5eCkHwU9",
                            "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"
                        },
                        "sequenceNumber": 0,
                        "zkappAccountUpdateIds": [
                            23,
                            24
                        ],
                        "zkappEventElementIds": [
                            19
                        ]
                    },
                    {
                        "data": [
                            "55456",
                            "1",
                            "86171",
                            "476632679643188528985438056980162363837707337312210785567256027534077014652",
                            "0"
                        ],
                        "accountUpdateId": "24",
                        "transactionInfo": {
                            "status": "applied",
                            "hash": "5Ju8GoXZWFqBKTAv3b1iMbc77yUhZEoVc5RwD5aj3VQw5eCkHwU9",
                            "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"
                        },
                        "sequenceNumber": 0,
                        "zkappAccountUpdateIds": [
                            23,
                            24
                        ],
                        "zkappEventElementIds": [
                            20
                        ]
                    }
                ]
            },
            {
                "blockInfo": {
                    "stateHash": "3NLEZvkc7Z1mRdFkh1bSppnKo8aZrB6VGe83J7UdKpL4PuLjtjqB",
                    "timestamp": "1729695707000",
                    "height": 70,
                    "parentHash": "3NKZqA81s8iHMQc4vcJxrcx6epVB7C9vBD8oV7K3EpCZzvfebCtP",
                    "chainStatus": "canonical",
                    "distanceFromMaxBlockHeight": 35,
                    "globalSlotSinceGenesis": 138
                },
                "actionState": {
                    "actionStateOne": "7365288285578028013264727183932035337741887015276045401228215954698411077529",
                    "actionStateTwo": "25619375377485241455325506385026113155326956491463643967090964080969742671789",
                    "actionStateThree": "25079927036070901246064867767436987657692091363973573142121686150614948079097",
                    "actionStateFour": "25079927036070901246064867767436987657692091363973573142121686150614948079097",
                    "actionStateFive": "25079927036070901246064867767436987657692091363973573142121686150614948079097"
                },
                "actionData": [
                    {
                        "data": [
                            "73850",
                            "1",
                            "11895",
                            "8253926607854573680301530978054957179571865656842078211022112667228198387920",
                            "1"
                        ],
                        "accountUpdateId": "33",
                        "transactionInfo": {
                            "status": "applied",
                            "hash": "5JuunjPt2xffChmddFDzdreN3TtTsADs1AiW4cgPXkYoFERyKPAi",
                            "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"
                        },
                        "sequenceNumber": 3,
                        "zkappAccountUpdateIds": [
                            33,
                            34
                        ],
                        "zkappEventElementIds": [
                            29
                        ]
                    },
                    {
                        "data": [
                            "73851",
                            "1",
                            "11895",
                            "8253926607854573680301530978054957179571865656842078211022112667228198387920",
                            "1"
                        ],
                        "accountUpdateId": "34",
                        "transactionInfo": {
                            "status": "applied",
                            "hash": "5JuunjPt2xffChmddFDzdreN3TtTsADs1AiW4cgPXkYoFERyKPAi",
                            "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"
                        },
                        "sequenceNumber": 3,
                        "zkappAccountUpdateIds": [
                            33,
                            34
                        ],
                        "zkappEventElementIds": [
                            30
                        ]
                    },
                    {
                        "data": [
                            "73237",
                            "1",
                            "52294",
                            "476632679643188528985438056980162363837707337312210785567256027534077014652",
                            "0"
                        ],
                        "accountUpdateId": "31",
                        "transactionInfo": {
                            "status": "applied",
                            "hash": "5JtuWFstMC6m3wgeoX8E5Nh1ZdwRdGXvhcrVbGB247Ph72KGpL23",
                            "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"
                        },
                        "sequenceNumber": 1,
                        "zkappAccountUpdateIds": [
                            31,
                            32
                        ],
                        "zkappEventElementIds": [
                            27
                        ]
                    },
                    {
                        "data": [
                            "73238",
                            "1",
                            "52294",
                            "476632679643188528985438056980162363837707337312210785567256027534077014652",
                            "0"
                        ],
                        "accountUpdateId": "32",
                        "transactionInfo": {
                            "status": "applied",
                            "hash": "5JtuWFstMC6m3wgeoX8E5Nh1ZdwRdGXvhcrVbGB247Ph72KGpL23",
                            "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"
                        },
                        "sequenceNumber": 1,
                        "zkappAccountUpdateIds": [
                            31,
                            32
                        ],
                        "zkappEventElementIds": [
                            28
                        ]
                    },
                    {
                        "data": [
                            "35093",
                            "1",
                            "39852",
                            "4611450100234868895111341341933539196411650948711574062334330094836938200670",
                            "1"
                        ],
                        "accountUpdateId": "29",
                        "transactionInfo": {
                            "status": "applied",
                            "hash": "5JtrkeMGmJFJQkVk27EtUsmzkLeiCinetkiwvVBfuHXrMqrwJvj2",
                            "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"
                        },
                        "sequenceNumber": 0,
                        "zkappAccountUpdateIds": [
                            29,
                            30
                        ],
                        "zkappEventElementIds": [
                            25
                        ]
                    },
                    {
                        "data": [
                            "35094",
                            "1",
                            "39852",
                            "4611450100234868895111341341933539196411650948711574062334330094836938200670",
                            "1"
                        ],
                        "accountUpdateId": "30",
                        "transactionInfo": {
                            "status": "applied",
                            "hash": "5JtrkeMGmJFJQkVk27EtUsmzkLeiCinetkiwvVBfuHXrMqrwJvj2",
                            "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"
                        },
                        "sequenceNumber": 0,
                        "zkappAccountUpdateIds": [
                            29,
                            30
                        ],
                        "zkappEventElementIds": [
                            26
                        ]
                    }
                ]
            }
        ]
    }
}

45930 commented 1 day ago

It looks like all the data is there, but it can be better-organized.

@deepthiskumar, do you know the relationship between events and actions? It seems like each transaction here has one event id, and 2 action ids. My test script does generate 2 actions per transaction, so I expected that, but I'm not sure why only one event would be sent.

45930 commented 1 day ago

Also it needs block-height (assuming the API returns canonical blocks only) to disambiguate transactions with same sequence numbers but are part of different blocks

This is already available, as blockInfo.height. In my example JSON, you can see the blocks 69 and 70 are present in the response. Within each block, there are transactions with sequence [0, 1, 2] on one, and [0, 1, 3] on the other. Then within each transaction, there are 2 events each. The accountUpdateId property is on the action itself, and the zkappAccountUpdateIds is an array showing the correct order on the transactionInfo.

I could add a property, accountUpdateIndex and remove the property accountUpdateId and handle that logic in the resolver, or we could send both the accountUpdateId of the action, and the zkappAccountUpdateIds of the transaction and allow clients to do the logic on their side.

45930 commented 23 hours ago

So we could have a response like

{
    "data": [
        "24651",
        "1",
        "63073",
        "4611450100234868895111341341933539196411650948711574062334330094836938200670",
        "1"
    ],
    "accountUpdateId": "28",
    "transactionInfo": {
        "status": "applied",
        "hash": "5JvAynd7NAHEiMqCpgZD85WCzLniBv18eSEJyuvNVK8pQL8Fi5kx",
        "memo": "E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH",
        "sequenceNumber": 2,
        "zkappAccountUpdateIds": [
            27,
            28
        ],
        "zkappEventElementIds": [
            24
        ]
    }
},

We have enough info here to tell us within the block that it's in, it belongs to transaction 2, and it is the second update in that sequence. That seems good enough to me, as long as that is the correct way to order things.

45930 commented 23 hours ago

For pending blocks, it looks like we follow the longest chain rule here:

SELECT
  id, state_hash, parent_hash, parent_id, height, global_slot_since_genesis, global_slot_since_hard_fork, timestamp, chain_status, ledger_hash, last_vrf_output
FROM
  blocks b
WHERE
  height = (SELECT max(height) FROM blocks)

We can probably handle filtering for canonical blocks in o1js. For reducing actions, I think we can ignore pending blocks by default, but for random exploratory queries, people might be interested to see pending data as well.

deepthiskumar commented 16 minutes ago

@deepthiskumar, do you know the relationship between events and actions? It seems like each transaction here has one event id, and 2 action ids. My test script does generate 2 actions per transaction, so I expected that, but I'm not sure why only one event would be sent.

Those are two separate fields in the transaction. Events are more for recording information corresponding to an account update. They don't change any on-chain state. Actions on the other hand are committed to the account (action_state field in an account). They

Can you generate at least two account updates (excluding the fee payer account update) each having two or more events and two or more actions? I think that should cover all the cases.

o1-labs / Archive-Node-API

adjust query to support ordering #107