search

rpcpool / yellowstone-grpc

solana geyser grpc service
GNU Affero General Public License v3.0
241 stars 102 forks source link

Transaction commitment #20

Closed losman0s closed 1 year ago

losman0s commented 1 year ago

Problem

I am using the gRPC geyser service as data source to index transactions, and for that it is important I trust that the transactions I receive actually make it into the block. However I notice that my DB has transactions that cannot be found on-chain.

My understanding from our discussion is that the intent is to let users cross-reference the transaction sub updates with the slot sub updates (giving the corresponding slot commitment) client-side, to know the txs' commitments. However afaik this makes the assumption that no block is uncled == any transaction received (i.e. processed) "makes it".

At the moment I need to perform an unsightly verification step to avoid storing wrong data, which is not ideal.

Proposal

I haven't dug into the server codebase, but I'm hoping there is a way for you to move the commitment matching to the source, therefore allowing transaction subscriptions with a specific commitment.

And let me know if I simply oversaw something, of course 😄

fanatid commented 1 year ago

I think in general it's possible to implement commitment on the server side but would require much more logic :thinking:

linuskendall commented 1 year ago

If you process the slot updates then you can detect and throw away uncles, by just following the slotNotification and the corresponding slot statuses.

You could for example, on any rooted slot notification delete transactions that are for a slot < the current rooted notification but where the slot isn't yet rooted.

Another way we might solve this in the plugin more easily would be to add a "uncle" notification so that the plugin keeps track of the slots and lets you know whenever a slot should be deeemed uncled?

linuskendall commented 1 year ago

The one challenge with sending transactions on commitment levels > processed is that the plugin will buffer the transactions until the commitment level for that particular slot is reached. It is definitely possible, but it adds another layer of potential failure and you would also get transactions notified all at once when that commitment level is reached.

losman0s commented 1 year ago

If you process the slot updates then you can detect and throw away uncles, by just following the slotNotification and the corresponding slot statuses.

Hmm yes that is true. Basically buffer on my side until the corresponding slot has been confirmed and only then store. Will give that a try, thanks!

I think in general it's possible to implement commitment on the server side but would require much more logic

Definitely not looking to bloat the server, the above should do it 👍