RCP-240731A: Improvements to consensus ordering of state processing

[dr-orlovsky]

authors: @dr-orlovsky, @zoedberg
layers: consensus
breaking: consensus, api, serialization

Background

RGB contracts have global state; which is assembled from the contributions of individual operations (genesis, state transitions and state extensions). Since schema defines a maximum number of global state items of each type, the ordering of this processing affects what is reported to the clients. The global state is also accessible via AluVM op codes (CnC, LdC) and can be used in validating consequent operations. Thus, the order in which operations are processed during the validation and state computing affects consensus and validity of future operations. This order is named "consensus ordering" (of operations or global state).

At this moment the ordering happens according to the following ruleset:

• genesis always goes first;
• we order the rest of operations basing on the mining height of a witness bitcoin transaction committing to the operation data;
• if the operation is not mined (i.e. its witness transaction is in mempool, RBF, lightning channel), it is ordered after all mined operations;
• state extensions, which doesn't have direct witness transaction are ordered using witness transaction of a first state transition which closes a single-use seal defined in the extension; and they always are processed before that state transition;
• if two or more witness transactions are mined in the same block, they are ordered by a lexicographic ordering of their txids;
• we always use block timestamp and not block height, such that we can support multiple layer 1 (bitcoin, liquid) with different block production times (and incomparable heights);
• for multiple off chain transactions (mempool, LN channel) we use a dedicated parameter priority provided as a part of their off chain status, such that HTLCs can be indicated to be processed after the commitment transaction.

Motivation

The provided ruleset has one inconsistency: in lightning channels we order HTLCs after commitment using a dedicated off chain priority field, which will not be reflected if these transactions get mined, when they will get re-ordered basing on their txids, such that there is no guarantee that the state of HTLCs will not come before the commitment transaction (which may render it invalid)

Proposal

It is proposed to:

• remove priority field from off chain transaction ordering;
• add new nonce field to state transactions and state transitions, such their operation id commits to it
• use that nonce for ordering operations with witnesses mined in the same block
• the value of nonce must be a 64-bit integer
• add operation type as a factor of ordering, assigning its higher priority than nonce.

This will allow to order graphs of off chain transactions in a consistent way, preserved even when they are mined.

Rationale

• 64-bit nonce. The max number of eltoo channel updates is 2^48 - 1. Since all updates are added to the graph, plus we need to order HTLCs (and we may have thousands of them), we need 64-bit nonce. Even though not of all of commitment transactions would be published, some may, and all of them must be ordered appropriately.
• operation type used in ordering. If we have two different state transitions of different mined into the same block, which (by mistake) have the same nonce, we still can order them using type. This allows usage of type id value as a way of setting strong ordering overriding nonce values (for instance token issues can be always ordered before spendings).

[zoedberg]

I'm not sure about this. First of all the statement:

there is no guarantee that the state of HTLCs will not come before the commitment transaction (which may render it invalid)

state transitions are explicit in which are the RGB inputs and associated UTXOs so I don't see how HTLCs could come before the commitment transactions.

Then, about this statement:

This will allow to order graphs of off chain transactions in a consistent way,

I'm not sure we actually need ordering offchain transactions. All commitment TXs will be spending the funding TX and RGB should not care about the order of these commitment TXs, it should be agnostic about the LN. Moreover as we discussed in https://github.com/RGB-WG/rgb-std/issues/238 the optimal solution for LN would be to avoid consuming the fascia for every channel update, since that would make the stash grow a lot over time. Initially you proposed to go with the suboptimal solution (i.e. consuming the fascia at every channel update) because it seemed faster to implement, but now it seems it's requiring a lot of work too. So I would reconsider the idea and instead implement the optimal solution (i.e. deterministically generate state transitions matching previous state). With that in place all the discussion about offchain TXs ordering should become irrelevant.

[dr-orlovsky]

state transitions are explicit in which are the RGB inputs and associated UTXOs so I don't see how HTLCs could come before the commitment transactions.

basing on the current RGB Core code :)

I'm not sure we actually need ordering offchain transactions.

Consensus uses this information, so it is needed. Otherwise the system won't work and will be buggy/attackable. This is not a question

it should be agnostic about the LN.

yes, it is agnostic. It has nothing to do with LN specifically, just any off chain transaction graph.

With that in place all the discussion about offchain TXs ordering should become irrelevant.

This is unrelated. It is relevant notwithstanding anything we do with specific LN details.

[zoedberg]

basing on the current RGB Core code :)

What do you mean?

Consensus uses this information, so it is needed. Otherwise the system won't work and will be buggy/attackable. This is not a question

IMO it would be useful if you shared more information on this, what attack/bug would be possible?

[dr-orlovsky]

What do you mean?

I mean that RGB Core uses consensus ordering inside AluVM and for organizing global state reported to the clients. It is part of the existing code base.

IMO it would be useful if you shared more information on this, what attack/bug would be possible?

I gave a full description in the proposal, "Motivation" section. Once mined, the ordering of the state processing may change, leading to invalid contracts which were already accepted (or resulting from an uncooperatively-closed channels)

[dr-orlovsky]

Consensus ordering was introduced in https://github.com/RGB-WG/rgb-core/issues/117 (those days global state was named metadata) and also described in the glossary https://github.com/orgs/RGB-WG/discussions/52 (see "consensus order").

[zoedberg]

I think there's been a misundestanding. Let's recap the conversation about HTLC ordering:

there is no guarantee that the state of HTLCs will not come before the commitment transaction (which may render it invalid)

state transitions are explicit in which are the RGB inputs and associated UTXOs so I don't see how HTLCs could come before the commitment transactions.

basing on the current RGB Core code :)

What do you mean?

I mean that RGB Core uses consensus ordering inside AluVM and for organizing global state reported to the clients. It is part of the existing code base.

From this discussion it seems RGB core is already able to order transactions, therefore I fail to understand the "Motivation" part of this RFC.

I understand ordering of onchain transactions is not the same of ordering offchain transactions, where you cannot use the height to order TXs. But 1) this could happen also onchain, think about CPFP and 2) it's always possible to order TXs following the inputs/outputs chain (in other words TXs can be ordered by which inputs are spending which outputs). So to me it's impossible that HTLC TX can "come before the commitment transaction" since the input of the HTLC TX is one of the UTXOs created by the commitment TX.

Once mined, the ordering of the state processing may change, leading to invalid contracts which were already accepted (or resulting from an uncooperatively-closed channels)

As just explained, I don't see how the order may change. But maybe I'm failing to understand what you're trying to say. Please try to explain this again, giving more details

[dr-orlovsky]

From this discussion it seems RGB core is already able to order transactions, therefore I fail to understand the "Motivation" part of this RFC.

It is not. It has no idea of bitcoin transaction ordering, and should not has. Otherwise it has to be re-designed embedding (partially) Bitcoin Core in itself.

That is what I say: the current RGB Core code has no idea of bitcoin transaction ordering. Since there is no 1-to-1 relation between bitcoin transaction and state transitions, even if it had, it wouldn't help.

So the issue is the state transitions ordering. And without the proposed consensus rules there is no way how to order them to match the lightning channel structure (and some other cases, so this is lightning-agnostic problem, just highlighted by lightning case)

[dr-orlovsky]

I understand ordering of onchain transactions

You constantly confuse bitcoin transactions with RGB state transitions. They are not matching each other, not 1-to-1.

[zoedberg]

I think I understood the issue. In https://github.com/RGB-WG/rgb-core/issues/117 you say:

the order of state transition follows the bitcoin consensus ordering of witness transaction, i.e. using block height and position in the block of the witness transaction

therefore RGB doesn't order onchain TXs based on inputs/outputs relations but based on TX height and TX position in the block, which means that it's impossible with current code to order offchain transactions, since they don't have a height nor position in the block.

Now I understand the necessity to find a solution to order offchain TXs.

In the nonce solution, though, I'm worried that we could have issues in restoring the used nonce in case of re-construction of a previous state (e.g. in LN when an attacker publishes a TX associated to an old channel update). But maybe I didn't understand the proposal. Could you please expand this part:

add new nonce field to state transactions and state transitions, such their operation id commits to it

?

Would the nonce be a random number or a number the dev will be able to set? How would that work in case multiple transitions need to be "committed to" (spent)?

[dr-orlovsky]

therefore RGB doesn't order onchain TXs based on inputs/outputs relations but based on TX height and TX position in the block,

Correct. The problem is that bitcoin transactions and RGB state transitions have many-to-many relationship, and one can't deterministically/unambiguously reconstruct an ordering for RGB state transitions just from how blockchain transactions are ordered - not even touching the case of mempool and state channels. And RGB state transitions are DAG, meaning that there is no linear order which can be always build from the DAG itself (which is a 2D net). We can't use a flow of seals (both in RGB transitions and bitcoin transactions) due to this reason: you have a DAG of seals and no linear ordering is possible. Using witness transaction ordering is the only thing which may work, but still we need something on top of it to solve ambiguity (since there is no bitcoin ordering for witness transactions in the same block).

Now I understand the necessity to find a solution to order offchain TXs.

Without that solution contract has no defined state, and all contract state introspections op-codes (reading global state) are impossible - and without those op codes and global state no DeFi apps can be made with RGB.

In the nonce solution, though, I'm worried that we could have issues in restoring the used nonce in case of re-construction of a previous state (e.g. in LN when an attacker publishes a TX associated to an old channel update).

The nonce becomes part of the state transitions, it is there - no need to restore anything:

add new nonce field to state transactions and state transitions, such their operation id commits to it

Would the nonce be a random number or a number the dev will be able to set?

Consensus doesn't put any requirements on the value of the nonce other than it is used for the ordering. I doubt any application can use random numbers to do the ordering :)

How would that work in case multiple transitions need to be "committed to" (spent)?

I do not understand the way you use the terminology. How you relate commitments and spends??

[dr-orlovsky]

Maybe it would be easier to read how the ordering works from the code itself: https://github.com/RGB-WG/rgb-core/blob/d06c627c846e9111a940aa84a8d5a670317d1f40/src/vm/contract.rs#L364-L404

since this structure has automatic Ord derive, the moment you put RGB operations into a BTreeMap<OpOrd, OpRef> you get consensus ordering automatically. The order of the enum variants and fields inside them is the order in which the values are compared by rust - as described in the doc comment.

The nonce value is taken from the operations: https://github.com/RGB-WG/rgb-core/blob/d06c627c846e9111a940aa84a8d5a670317d1f40/src/operation/operations.rs#L229-L231; each operation has nonce in its own data: https://github.com/RGB-WG/rgb-core/blob/d06c627c846e9111a940aa84a8d5a670317d1f40/src/operation/operations.rs#L385 (nonce of genesis is always u8::MAX: https://github.com/RGB-WG/rgb-core/blob/d06c627c846e9111a940aa84a8d5a670317d1f40/src/operation/operations.rs#L520-L521)

Operation ids commit to the nonce value alongside all other data they have, so the nonce can't be changed: https://github.com/RGB-WG/rgb-core/blob/d06c627c846e9111a940aa84a8d5a670317d1f40/src/operation/commit.rs#L310

[zoedberg]

I don't have the time to look at the code right now, I will do it on Monday. I'm not sure about this though:

The nonce becomes part of the state transitions, it is there - no need to restore anything:

it might become an issue in the way we think LN should work. In LN we would like to be able to avoid saving a bunch of data to reconstruct the RGB information to re-color previous commitment TXs. The desired workflow would be that we just save the RGB amounts that were assigned to the commitment TX vouts and then deterministically recreate the Fascia that was alredy created in the past but not stored nor consumed. This way we would reduce the space required for each channel update at a minimum. If we are not able to reconstruct the nonce deterministically then I'm afraid we will not be able to achieve our desired behavior.

[dr-orlovsky]

I do not see how this can interfere with LN. Maybe you keep thinking that nonce is random, but I already pointed out that random nonces wouldn't work since you can't get deterministic ordering with them. Thus, nonce it is deterministic. QED

[zoedberg]

Maybe you keep thinking that nonce is random, but I already pointed out that random nonces wouldn't work since you can't get deterministic ordering with them.

I'm not thinking it's random, I'm thinking it's not deterministic in the context of LN channel updates. I'll rephrase what we would like to achieve:

For each LN channel update we want to save only the RGB amounts assigned to each commitment TX vout (we don't want to save the nonce too).

To achieve this the nonce needs to be the same for all commitment TXs (e.g. 1) and the same for all HTLC TXs (e.g. 2) (examples are assuming an ascending ordering, but since you've set the genesis to u8::MAX it will probably be descending order, anyway I think it's clear what I'm saying). But we are not sure this will be the case because of:

• what you've written here isn't clear about this
• the comment in the code saying If two or more operations share the same witness transaction ordering, they are first ordered basing on their `nonce` value, and if it is also the same, basing on their operation id value.
• nonce meaning (i.e. number used only once)

The code you've linked unfortunately doesn't answer my question (i.e. who/where the nonce is "decided").

Anyway, I looked at the code myself and I've found what I think is a bug:

by following the code from OpOrd I've found GlobalOrd::transition which is called only by MemContract<M>::global with a nonce coming from MemGlobalState::known, which gets filled by MemContractState::add_operation, called by MemContractWriter<'mem>::add_transition, called on bundle.known_transitions, which comes from the Psbt::rgb_bundles method, that gets filled with transitions that are retrieved by Psbt::rgb_transition that retrieves the transitions inserted in the proprietary map by the Psbt::push_rgb_transition method, which gets the transition from the Psbt::rgb_embed method that gets the Batch retrieved from stock.compose, that constructs the transitions via the TransitionBuilder, retrieved through the stash.transition_builder method that uses TransitionBuilder::named_transition or TransitionBuilder::default_transition (https://github.com/RGB-WG/rgb-std/blob/develop/src/persistence/stash.rs#L348-L366). Both of these, TransitionBuilder::default_transition (https://github.com/RGB-WG/rgb-std/blob/develop/src/interface/builder.rs#L463-L476) and TransitionBuilder::named_transition (https://github.com/RGB-WG/rgb-std/blob/develop/src/interface/builder.rs#L493-L506) set the nonce to u8::MAX.

TLDR: it seems all transitions use the same nonce (u8::MAX)

[dr-orlovsky]

TL;DR: No,

• there is no bug, the code Is correct;
• this proposal doesn't affect determinism of LN and adds nothing to make re-generation of all state transitions for past channel state impossible.

---

The proposal defines consensus part of the nonce, meaning how it is validated and not how it is used. The use of nonce is the choice of specific client library; and the implementations are free to choose any way of using it unless this use invalidates the state transition basing on the consensus rules.

Specifically, nonce defines ordering of state transitions if their respective witness transactions are mined in the same bitcoin block - or when they are still in the mempool. Thus, if the state transitions are unrelated, i.e. one of them doesn't spend outputs of the other and come from different wallets, the nonce will affect the ordering of the global state, important for some DeFi protocols (but not for simple RGB20 cases). RGB standard library assumes game-theoretical model that each participant would like to get its state transition ordered last (i.e. preserved in a global state scope longer than others), using by default the maximum value, which is u8::MAX. In that case, they will be ordered by their operation ids, as said by the publication.

On the other hand, when we have a state channel, like in lightning, we need to make sure that related state transitions (meaning state transitions spending outputs of each other, like HTLCs spend commitment outputs in LN) we should not use the same nonce for them (and, hence the name for this value, nonce - number used once). The custom nonce can be set done using TransitionBuilder::set_priority API, present in the standard library.

Which specific value should be used in case of LN? While there is no consensus-rule for that and there might be different approaches to the question, the one that makes most of sense for me is the following:

• set u8::MAX to the state transitions which respective witness transactions are tips in the channel transaction graph (these are HTLCs, in case of BOLT Lightning)
• reduce nonce value by 1 for each state transaction which precedes the tip by one in the off chain graph (i.e. in case of BOLT LN, set it for commitment state transition to 0xFE).

This approach guarantees that global state of HTLCs spending commitment outputs are always accounted after the global state of commitment state transition. Yes, if a multiple HTLCs are present, they would share the same nonce, but this is not a problem, since they belong to different branches of the off chain graph (and will be ordered basing by their op ids, which is fully deterministic).

One may choose some different approach, but with nearly any approach (unless you are using random numbers, which will break the consensus meaning of nonce) you just can't affect the determinism of re-generating state transitions for LN channel: apply the same logic to each channel state update, and you have your re-generation determinism.

[zoedberg]

Now it makes sense, thanks for explaining this in detail.

There's no TransitionBuilder::set_priority method, but I've found Batch::set_priority. I didn't notice that method but with that I agree there is no bug.

[dr-orlovsky]

There's no TransitionBuilder::set_priority method, but I've found Batch::set_priority. I didn't notice that method but with that I agree there is no bug.

Transition builders has method set_nonce.

But the idea is that all state transitions for the same witness transaction (including blank ones) should have the same priority, so it is advised to use batch set_priority method and not just individual transition builder.

---

A question: right now we use nonce as u8 value, and set different values basing on the depth of transaction in the state channel graph. This means that we may have up to 256 levels of depth, which seemed enough for all future use cases (LN BOLT uses only 2).

However, on each depth level state transitions will be ordered basing on their op ids, i.e. deterministically, but arbitrary. What I mean is that if we have 1000 HTLCs, they will share the same nonce and their ordering will have no meaning. Wouldn't it be better to order them basing on the time when they were added to the channel? In this case we need to change nonce from u8 to u16 or even u24 and use something like nonce = (depth << 16) | htlc_ord

[zoedberg]

Wouldn't it be better to order them basing on the time when they were added to the channel?

I don't see why it would be better. Could you please share the use case or benefit of ordering them?

Another consideration instead: wouldn't it be more handy if we use 0 instead of u8::MAX and for example in LN instead of setting the commitment to 0xFE and the HTLCs to OxFF we would set the commitment to 0 and the HTLCs to 1? This way in a non-LN setup we can add more offchain TXs later on, without the need to reconstruct all the previous ones.

[dr-orlovsky]

I don't see why it would be better. Could you please share the use case or benefit of ordering them?

Let's assume we have a lightning channel operating DeFi protocol - for instance a lending, something alike AAVE. It makes a little sense for the ancient BOLT balkanized by Lightning Labs, but quite a lot for some multi-peer channels like Nucleus, which eventually become part of the LNP generalization of state channels.

In such a channel, state transitions will create new global state, updating the information about the number of the global values, like LTV etc. Such global state will be defined with max_values: 1 meaning that only the most recent value is valid (and the rest of the previous history is discarded during the validation). Thus, ordering of state transitions is important, since if some state transition ordering is changed, it becomes invalid (operating with invalid LTV, which may mean liquidation etc). In such cases, HTLCs, which spend commitment, must be ordered by the time they were agreed between parties, and not arbitrary opid values.

I understand that you may say "ancient BOLTs doesn't need that advancement", but since it is nearly zero-effort better to do things properly from the start, even though nobody will use it in that form and everybody will switch to Nucleus and LNP, when available :)

Another consideration instead: wouldn't it be more handy if we use 0 instead of u8::MAX and for example in LN instead of setting the commitment to 0xFE and the HTLCs to OxFF we would set the commitment to 0 and the HTLCs to 1?

I do not see why and how that is more handy: it takes the same amount of code to say x = 1u8 or x = 0xFFu8 (yes, one need to add u8 suffix to make sure that when the type would change you get compiler notifying you to double-check that assignment whether it still makes sense).

My logic is that in decentralized protocols everybody would compete to become the "last value" (see example with decentralized lending above). Thus, if you put 0x00 instead of 0xFF somebody will just fork your code and use 0xFF instead. The rule I proposed is not a consensus rule, but a standardness rule saving us from proliferation of the forks.

[zoedberg]

I understand that you may say "ancient BOLTs doesn't need that advancement", but since it is nearly zero-effort better to do things properly from the start, even though nobody will use it in that form and everybody will switch to Nucleus and LNP, when available :)

If this just requires changing the nonce from a u8 to a u16 (or greater) I don't see any issue. Actually, if I correctly recall how eltoo works I think we need it to be greater (probably even u64, since the max number of channel updates is 2^48 - 1). In "ancient" LN we won't use the depth since it's not required but I agree we should already take future use cases into account.

I do not see why and how that is more handy: it takes the same amount of code to say x = 1u8 or x = 0xFFu8 (yes, one need to add u8 suffix to make sure that when the type would change you get compiler notifying you to double-check that assignment whether it still makes sense).

My logic is that in decentralized protocols everybody would compete to become the "last value" (see example with decentralized lending above). Thus, if you put 0x00 instead of 0xFF somebody will just fork your code and use 0xFF instead. The rule I proposed is not a consensus rule, but a standardness rule saving us from proliferation of the forks.

I think you missed my point. Using a progressive nonce starting from 0 allows to build offchain TX chains whose lenght is not known in advance (without the need of recreating past TXs when a new one occurs at the end of the chain). Moreover, even if the nonce doesn't get considered in case of mined TXs, I think that's counterintuitive that they have the nonce set to u8::MAX, since ordering is ascending and u8::MAX would mean "consider this TX as the latest of the ordered chain". Anyway I see that's not a consensus rule so this discussion can end here.

[dr-orlovsky]

Actually, if I correctly recall how eltoo works I think we need it to be greater (probably even u64, since the max number of channel updates is 2^48 - 1).

But we do not need to increase the nonce on each channel update, we need to increase it on each tx depth in the graph of a specific channel state. Eltoo channel transaction graph can be deeper than in LN (in some variants), but it would never exceed 64k depth - otherwise it won't be realistically enforceable onchain (it would take ages and fortune to publish >64k bitcoin transactions just to uncooperatively close the channel) :)

In "ancient" LN we won't use the depth

We still need it to order HTLCs after the commitment transaction. There might be some limited applications requiring global state update, like with engravings in NFTs (for some LN-based games etc).

Using a progressive nonce starting from 0 allows to build offchain TX chains whose lenght is not known in advance (without the need of recreating past TXs when a new one occurs at the end of the chain).

Using 0 for nonce you provide unchain actors with an opportunity to front-run a LN-based RGB app. But yes, this is not a consensus question.

[dr-orlovsky]

Giving it a second thought I have to admit that @zoedberg is right: to properly support eltoo we need u64. The reason is that while we would never publish the whole of the eltoo offchain graph into mempool or onchain (and only few transactions even in uncooperative closing will hit the chain), they all are still have to be ordered consenquently, and there is no way of doing that unless incrementally increase nonce with each channel update.

Implemented in https://github.com/RGB-WG/rgb-core/pull/267

[zoedberg]

We still need it to order HTLCs after the commitment transaction.

We'll need the nonce for that, not the depth

Using 0 for nonce you provide unchain actors with an opportunity to front-run a LN-based RGB app.

I'm not sure what do you mean here, could you please clarify?

[St333p]

I'm a bit confused on how transactions in the same block should be ordered:

• this specification says:

  if two or more witness transactions are mined in the same block, they are ordered by a lexicographic ordering of their txids;

• in https://github.com/RGB-WG/rgb-core/issues/117 you wrote:

  using block height and position in the block of the witness transaction

• even this might not be correct in all cases. Bitcoin consensus ensures that chained transactions in a block are ordered correctly within the block, but RGB blinded transfers are not necessarily related at the bitcoin level.

Is it enough to have a deterministic ordering or do we need to ensure that this ordering always coincides with the RGB DAG one? In other words, are there major issues if two chained operations (the second spends an allocation created by the first) are ordered backwards (second before first)?

[dr-orlovsky]

See https://github.com/RGB-WG/rgb-core/pull/275#discussion_r1800860496

At this moment there is no written specification for the ordering; this specific RCP was proposing nonce, and no RCP were written for the change from txid to opid for ordering within the block. Once we finalize the v0.11 the whole algorithm would need to be written as a RFC.

even this might not be correct in all cases. Bitcoin consensus ensures that chained transactions in a block are ordered correctly within the block, but RGB blinded transfers are not necessarily related at the bitcoin level.

Not sure I understand: blank state transitions are state transitions, which do have witness transactions, and they are related to the bitcoin blocks the same way as normal ones.

are there major issues if two chained operations (the second spends an allocation created by the first) are ordered backwards (second before first)?

Good question! That's why we have introduced nonce in this specific RCP. For instance, it is the way we order in LN

[St333p]

Not sure I understand: blank state transitions are state transitions, which do have witness transactions, and they are related to the bitcoin blocks the same way as normal ones.

I never mentioned blank transitions. Let's say we have two transactions, one assigning assets to a blinded UTXO and one spending that UTXO, both mined in the same block. The second one can appear in a block before the first one without breaking bitcoin consensus, but it would not be the expected order from RGB point of view. Can this cause issues?

[dr-orlovsky]

Nope, they can't appear in that sequence. If transition 2 spends outputs of transition 1, than it means its witness transaction spends utxos which were defined as seals in transition 1. Thus, bitcoin consensus would prevent witness 2 to be mined before witness 1.

[St333p]

Transition 1 may be allocating assets to an existing blinded UTXO, which was created by a transaction mined in a previous block. Bitcoin consensus does not even know that transition 1 is defining a seal on that UTXO, am I missing something?

[dr-orlovsky]

Transition 2 spends outputs from transition 1, right? Even though they are assigned to UTXOs which are mined for a long time, transition 2 can't be valid unless transition 1 is valid; and transition 1 becomes valid only when its witness transaction is mined. Thus, witness transaction 1 has to be mined before witness transaction 2, and not a block or few after.

Yes, there might be a reorg which will put witness 2 into block n and witness 1 to block n+1, but if this would affect the contract it is up to the transaction producer to ensure that reorg in not possible (for instance waiting for 2 blocks before doing the second transition).

Yes, putting witness tx for transition 2 before the witness tx for transition 1 would change their RGB consensus ordering. I can theoretically imagine very wired validation script which will make transition 2 invalid in this case - but, if I said above, the user of such weird contract must wait for secure depth before doing transition 2.

RGB consensus ordering is not about deciding "the right order" (according to the contract rules). It is about using a deterministic ordering, which doesn't change once we are mined at sufficient depth where re-orgs would not happen.

Re-org can destroy transaction 2 anyway even without consensus ordering, and there is nohow we can prevent that from happening other than recommending users to transact in a safe manner.

[dr-orlovsky]

So, I will separate two independent issues.

First, "what happens if witness transaction for transition B spending output of transition A gets mined (or re-orged) before witness transaction A"? This question is not specific to consensus ordering and affects RGB contracts even if no ordering is applied. The answer is simple: it may not be a problem, or may cause problems, depending on the structure of transitions, validation scripts etc. The user (wallet) always knows when it is safe to do, and if it is not save, a user (wallet) must ensure that it waits enough blocks between publishing witness transaction A and B - or ensure that witness B spends from witness A transaction, taking bitcoin consensus into help.

Second, "what can we do to make RGB consensus ordering better"? Your idea was to take into account the DAG of RGB operations; but the problem here is that consensus ordering needs a linear sequence and not a DAG; and mathematically not each DAG can be deterministically turned into a sequence! Thus, in bitcoin we need PoW consensus for that, and in RGB we rely on bitcoin consensus as well + nonce mechanism + opid. Can we do better? I do not think so. We can chose a different tradeoff - like making nonce, defined by a user, to be the main ordering rule, having a higher priority over the block height? (i.e. abandon pegging to bitcoin consensus) Wall, this will decrease UX, and won't prevent all the problems (like problem one described in the previous paragraph). Thus, I think the way we order ops today is the best possible one.

[zoedberg]

@dr-orlovsky sorry for bringing this up again, after discussing this for a while, but could you please clarify your statement in the RCP description:

the order in which operations are processed during the validation and state computing affects consensus and validity of future operations

I'm starting to think that consensus ordering is not actually needed

[dr-orlovsky]

Consensus ordering is used to organize DAG of operations into a linear history in a deterministic way, which from there goes to validation and VM. VM reconstructs the state of the contract, against which each new operation is validated. If you do not have a linear order, you do jot have a contracts state.

[zoedberg]

Consensus ordering is used to organize DAG of operations into a linear history in a deterministic way, which from there goes to validation and VM

Current validation seems to not care about ordering of operations. Is this feature meant for future possible contracts?

If you do not have a linear order, you do jot have a contracts state.

From the code I see that operations are ordered only in the context of the validation, where ordering doesn't seem to affect the contract state.

[dr-orlovsky]

Current validation seems to not care about ordering of operations. Is this feature meant for future possible contracts?

No idea how you have got that impression. It heavily rely on it and can't be done in any other way, once you get the concept of a global contract state (which I was finally able to complete this summer)

From the code I see that operations are ordered only in the context of the validation, where ordering doesn't seem to affect the contract state.

This is not true. You probably read something else, and not the current RGB consensus validation code

[zoedberg]

Please run the tests applying this commit to rgb-core https://github.com/zoedberg/rgb-core/commit/60356bcce142e36dfa0748cc877c177650952731 (this has been made on top of develop but you can see this also if you put it on top of beta 8)

You'll see that

• the Ordering WitnessPos log appears only between logs order start and order end logs, proving that there are no other places were ordering code is used
• shuffling the operations in random order before running the validation will not make any test fail

I also removed usage of nonce in both RLN and rgb-tests and also there I've seen no issue, the HTLC transaction can be constructed and there are no validation issues at any point in time.

Hence I think consensus ordering is not needed and validation is already able to validate operations based on the DAG of RGB operations, which can be considered linear in the context of "taking a terminal operation and validate every operation from there to the genesis and repeat this for every terminal I'm validating"

[dr-orlovsky]

As we discussed yesterday and I explained in our personal meeting,

1. Ordering works through the nature of BTreeMap and iterators. No specific calls for APIs from any RGB-related happen for that reason
2. Schemata we use today do not have a global state of more than 1 element, so the fact that their validation is not affected by the changes to ordering doesn't imply it is not working or being used

Writing it here for the community as a memo note

[zoedberg]

As we re-discussed in a follow-up meeting:

1. This was clear
2. From what I understood we cannot distinguish operations that affect the global state, thus we should validate all operations following the consensus ordering rules. But I may have misunderstood this part

Peeking at the new validation code from the future v0.12 we discussed the explicit check that has been added to make sure the given operations are ordered following the consensus ordering rules. This check may be too restrictive since it would break cases that were possible in v0.11, like:

• a chain of "blinded transfers" (where witness TXs are not connected to each other) followed by a re-org that inverts the order of the TXs and make RGB transitions have an order different from their witness TXs
• the example @st333p made: https://github.com/RGB-WG/RFC/issues/10#issuecomment-2414306850 (i.e. a chain of "blinded transfers" with witness TXs mined in the same block)

As discussed, waiting 6 confirmations after receiving on a blinded UTXO doesn't seem a viable solution, thus we should not enforce the ordering of witness TXs to be the same as the RGB transitions one (like it's happening in v0.11).

[dr-orlovsky]

From what I understood we cannot distinguish operations that affect the global state, thus we should validate all operations following the consensus ordering rules. But I may have misunderstood this part

You understood it right