RPC endpoint for transaction tracking by hash

[jacogr]

This is an oft-requested featured, both in the Element chats and issues. This comes up at least once every 2 weeks. (Latest version on the JS API here - https://github.com/polkadot-js/api/issues/2445)

So basically -

• author_subscribeExtrinsicStatus (extrinsicHash: Hash)
• tracks both the pool (as pending) and subsequent blocks
• returns the same status as we have in the submitAndWatch endpoint

cc @ngotchac

[NikVolf]

It was already implemented once and then reverted, we can probably revert the revert https://github.com/paritytech/substrate/pull/3873

[ngotchac]

This was an RPC to watch an extrinsic though (if I'm not mistaken). My take of the issue here is that there should be an RPC to get the status of an extrinsic, without having to watch it

[NikVolf]

But this is impossible without watching, substrate has no transaction index by hash

[ngotchac]

Yes, that's what I was told, but it would be nice to have such an index

[bkchr]

What is a transaction index by hash?

[ngotchac]

@bkchr I'm not sure I understand the question, but the idea is that from an extrinsic hash, you should be able to get one (or multiple) extrinsics. So the index would likely hold a list of blocks maybe where the extrinsic happened and the index of the extrinsic within the block ; does that make sense?

[bkchr]

Yeah makes sense.

But how should this work when a transaction hash is not unique?

[ngotchac]

Again, I don't know how indexes work right now. But the idea (as I said above) is to hold a list of blocks and (extrinsic) indexes

[bkchr]

Yeah I read what you have written :D

Why do we need to track an transaction by hash? What is the use case?

[ngotchac]

Ah sorry, I thought you meant implementation wise, but I guess the question was about the use-case. So I stumbled upon this issue when I had a process sending extrsinsics on-chain, writing in DB the extrinsic hash, but another process (a worker) who would watch for the extrinsic finality. The fact that it was in 2 separate processes made it impossible to submit and watch. Also, sometimes I only want to use the HTTP provider (reasons are out of scope here), and I don't think one can submit and watch with such a provider.

(and in this case, the fact that two extrinsics shared the same hash was not possible since it was a signed transaction)

[jacogr]

This comes up very often, I think a reason for this is that people are used to working with this functionality on other blockchains. For instance, people want to submit (non-subscription), get the hash and then track the extrinsic elsewhere.

My bigger worry is the following scenario - tx was submitted 20 blocks ago, already included, does this just report nothing? In the case of reporting "nothing" you can even emulate the tracking on the middleware layer - (a) read pending, (b) read blocks, report back when found (you obviously miss stuff like "Ready", "Broadcast", but ignoring that for now)

In the case of the above reporting "something" (i.e. about where it was included in the past), this is a lot more effort since there is no indexing. (And not sure we really want to add it either or actually can reliably do so). Having said that, even when the above "just check the hash" is available, this would probably be asked for next.

[ngotchac]

I'm not sure I understand why it would report nothing. The solution here would be to add such an index, from extrinsic hash to list of (block_hash, extrinsic_index) : is this not possible to implement?

The index would be updated when blocks are imported, and updated if a block gets removed/on forks.

For extrinsics not yet in blocks, but in the mempool/pending transactions pool ; it would just return the same info as for other included extrinsics, but with an empty _blockhash and _blockindex of -1.

So finally, the query will check for the mempool and the index, which shouldn't take too long I guess

[jacogr]

"nothing" is for past transactions where it is already included. (If you don't have more DB space allocated)

[ngotchac]

Right. So it would just re-build the index from scratch I guess ; but that'd be a one time operation.