Introduce v2 `Blob` type with verified signer

[cmwaters]

Summary

Parent issue: https://github.com/celestiaorg/celestia-app/issues/2977

This proposal is based on my understanding of a feature request from Sovereign Labs and a feature I believe would apply to all enshrined rollups: these are all rollups where one or more addresses are whitelisted and only blobs from those accounts are deemed valid.

Proposal

To implement this we would first need to modify the blob protobuf:

message Blob {
  bytes namespace_id = 1;
  bytes data = 2;
  uint32 share_version = 3;
  uint32 namespace_version = 4;
  string signer = 5;
}

Given proto's backwards compatibility, users could still submit the old Blob type and signer would just be an empty string. The new block validity rule would thus be that if the signer was not empty, then it must match that of the PFB that references it.

However, when decoding the Blobs from the Share format, older versions would ignore the signer regardless if it were filled or not. Thus for backwards compatibility for the read path we would need to ensure that Blobs that use the signer, must use the v2 namespace version. Older software would thus error about not supported namespaces version when trying to read from it but could still support the first blob version without needing to upgrade.

We may want to consider other data that the Celestia validators could verify as an additional service to the rollup. These however don't need to be bundled up with the signer addition and could simply be added later.

[musalbas]

Where is this Blob struct used - does this signer actually end up inside the dataRoot (and if so where - in the PFB struct, or in the blobs themselves) or only in raw Tendermint block data? As it needs to be the former to be usable by rollups

[nashqueue]

I am assuming that we populate the first share of a blob with the signer. Can we make it so that we populate it with any metadata to make it forward compatible ?

Then additionally could we sort it by the metadata inside the namespace ?

More detailed proposal is here: https://forum.celestia.org/t/customize-your-namespace-rollup-resource-pricing-account-abstraction-and-native-restaking/1666#intra-namespace-sorting-and-sub-namespaces-10

[cmwaters]

Nothing about where the blob is changes. It is still part of the square and thus part of the dataRoot. It is in the Blob itself. When you call GetAll(height, namespaces) on node the blobs it returns will have the signer.

I am assuming that we populate the first share of a blob with the signer. Can we make it so that we populate it with any metadata to make it forward compatible ?

Currently no, it's based on how protobuf encodes bytes (which I believe would be somewhere at the end). How would you construct it to be forwards compatible (i.e. where the client doesn't know the schema of the metadata)?

[musalbas]

What I'm confused about then, is that this blob struct isn't how the blob is actually serialized in the dataRoot, as for example, the namespace version and share version is encoded in at the start of each share, so perhaps it would be helpful to frame this proposal in terms of how it will be serialized in the block, which is what's relevant to based ZK rollups that wanted to parse the signer, as they need to verify blobs directly in the dataRoot, rather than how it's represented by APIs

Or to clarify are you proposing just keeping the share format the same, but have the API additionally return the signer? The issue with that then, is the signer is not authenticated when returned by API

[preston-evans98]

@musalbas is correct that we need the sender to be encoded in the first share of the blob for the Sovereign SDK. Assuming the ultimate design ensures that, this proposal looks great to me!

[cmwaters]

Okay, yes I should have also included a schematic of how this changes the share encoding format. The new format (for the first share) would thus be:

[nashqueue]

The graphic shows how I assumed that It would look like. My proposal is that instead of introducing a different type of blob submission encode the rule "Add signer to the first share" in the namespace .

This rule would apply now to the whole namespace guaranteeing that the signer is the first, but leaving this as optional.

Other metadata that the signer would want to know could be how much the user paid in gas or tip to post to Celestia. With these types of metadata, you can customize your namespace and do not need to read the PFB-Namespace for pfb-submission-related data.

We might also want to add the verification key of the snark account posting a blob for example.

So instead of creating a new namespace version for all permutations of things we might want to add I would suggest to have that be encoded in the namespace ID, as we have some room from the zeros

[cmwaters]

So if I understand you correctly, you're saying increase the size of namespace version to be more than 1 byte (currently 256 versions) and decrease the namespaceID from 28 bytes. This is for the sake of allowing more variations in the namespace rules

[nashqueue]

That could definitly be an option too. In my option the rules that you want to use should be endoded iside the namespace itself. So when I choose to post to a namespace ID with a special prefix lets say its starts with 1 then I know that the signer is in the first share of the blob. That way when submitting a blob I only have to use the correct namespace.

Maybe we are talking about the same thing and both solutions are isomorphic

[musalbas]

The suggested namespace format makes sense to me.

The only question is if we want to also add even more metadata, perhaps committed to as a Merkle root of a tree of metadata. However, perhaps this can be more for v3 Blobs, and we can keep v2 simple with just an addition signer field.

[nashqueue]

This is the first stab at making it forward-compatible.

The flow would be the following. Each time we change the namespace version we can add and modify how we use the encoding in the namespace rules.

One of the rules should be adding the signer. But that should be optional, up to the user, and not enforced in Celestia.

From the user's perspective, choosing a rule inside a namespace would be as simple as choosing a new namespace. We have 10 zero bytes that are up for grabs, and we can decide that the first bit is if we want a signer or not.

The rules could enforce anything. I would refer to the blog post again for that.

[musalbas]

I thought your original proposal was about making the rule programmable with a scripting or ZK system? If you're just proposing that we reserve 10 bytes but leave the future rules open to future upgrades, IMO this is the same as Callum's original proposal of just using namespace v2 for this, use namespace v3 and so on for future upgrades and rules.

[nashqueue]

Scripting or a ZK system could also be enabled by those 10 bytes. The only thing I am arguing for this post, in particular, is that adding the signer should be optional and not enforced for everyone.

I may have misunderstood the behavior of incrementing to a bigger namespace. Does the user decide which namespace version is used? If that is the case, I am all for it, as we would achieve the same effect as you said and can add the rules in the future for v3.

In the future, I would like to see other rules, such as limiting the size of the namespace / blobs and intra-namespace sorting. These are orthogonal to scripting / zk.

It would be great to document how an upgrade to a newer namespace would impact user experience / upstream issues.

[rootulp]

Does the user decide which namespace version is used?

Yes the user decides which namespace version they want to use when they submit a MsgPayForBlob. See NewMsgPayForBlobs which has a parameter for Blob where the user can specify a namespace version.

[cmwaters]

After some offline discussion, we're leaning towards using the share version to indicate the addition of the signer in the share format. Share version can also be set by the user in tx submission.

The current options are:

• Bump share version only.
• Bump share version and namespace version. The namespace version would have the rule that blobs to that namespace would have to have the new share version enforcing that all blobs to that namespace version have a signer.

[evan-forbes]

closing as completed