Higher upper limit of `GeneralKey` in XCMv3

[ghzlatarev]

Hi guys,

Thank you for the great work on XCM infrastructure and the attention on this post.

At Manta we have a question regarding the GeneralKey junction of multi locations in XCM v3 PR https://github.com/paritytech/polkadot/blob/6810d06af95324b10a822e995c5f806caabf2e7a/xcm/src/v3/junction.rs#L231 We would like use this junction to embed a zero knowledge proof, which is basically a public key of our ledger's private UTXOs. This seems to fit in the general use case of the junction. More concretely we want to send this ZKP along with ORML.xTokens transfers and use it on our chain to dispatch an extrinsic that will directly privatize the transferred tokens into our MantaPay ledger. This will allow us to hide the complexity from front-ends so they can easily do the transfer and privatization in a single call, think of a "private checkbox" functionality. All of the required logic will be limited to our own chain and how we interpret incoming messages.

However our problem is that the upper limit will only be 32 bytes, and our ZKP is 192 bytes. Is it possible to bump this value or perhaps add another junction with higher capacity ?

There probably are other use cases that could take advantage of a larger GeneralKey for their own needs. For reference we also have an alternative approach for this kind of functionality here https://github.com/open-web3-stack/open-runtime-module-library/issues/766. Feedback would be appreciated.

@KiChjang @gavofyork @shawntabrizi

[rphmeier]

Could this be solved using hashing in some way? I don't fully understand the control flow but you could probably maintain some map from UTXO proof public-keys to blake32 hashes, either on-chain or off-chain (by whoever submits these extrinsics)

[stechu]

@rphmeier sorry for lacking of context, here is the work flow (the UTXO is not very relevant here). Say a user of Acala want to privatize her aUSD on Manta. Currently, she need to send her aUSD from Acala to Manta, and then privatize the aUSD on Manta, which is fine but not ideal. The use case that we would like to enable is to "one click" privatize the asset for different parachains that we are serving (currently we have XCM with Moonbeam, Acala, Phala, Astar etc).

So the steps would be like below:

1. Acala (on user's behalf, say Alice) sends an XCM message containing both the aUSD token transfer request and zero-knowledge proof (192 bytes).
2. Manta, upon receiving Acala's message, directly mint the correct amount of paUSD (private aUSD) to Alice. This essentially calls to_private and feed it with the ZKP passed.

You can see currently, the only bottleneck of this implementation is that we don't have a proper way to putting zkp payload into XCM. I don't think hashing is a very good solution here since this requires a off-band communication mechanism and the payload here is not in particular large.

[xlc]

32 bytes is also not enough to hold the CurrencyId type for Acala. I understand we want to everything is bounded and in that case can we have GeneralKey64, GeneralKey128, GeneralKey256? So that we can actually fit meaningful amount of data inside.

[KiChjang]

So, the annoying thing about this is that this will bloat the MaxEncodedLen of anything that contains a MultiLocation, as it would take the largest variant of the Junction enum and calculate the storage proof size based on it, and as a result of that, we might grossly overestimate how much resource a single XCM execution would utilize, and thus harms the throughput of sending/receiving XCMs.

I haven't really got any good ideas to overcome this just yet, but I have to put this out here as a factor for consideration when it comes to introducing a Junction variant that has a size larger than what we currently have now.

[xlc]

It is simply impossible to limit the message size at compile time. How can you limit the size of Call in Transact without introducing significant breaking changes? We should just enforce a runtime size limit and fail it when detected big message.

[gavofyork]

MultiLocations are really not designed to contain arbitrary size data in our current model of weights v2.

Transact is rather a special case as it's only a single instruction and by definition it must support an arbitrary amount of data. I'm afraid a small limit is non-negotiable or we would end up with unbounded or huge MultiLocation/MultiAsset storage footprints everywhere they're used in storage (which is really a lot of places).

You could perhaps introduce a new XCM instruction (and perhaps initially just a transaction) which allowed the possibility of defining a preimage within the execution context. You could then use the hash of the proof as an identifier and look it up in the backend. This would allow the proof to be communicated over XCM without the need to have an unbounded MultiLocation type.

[stechu]

@KiChjang @gavofyork one work-around that we can think of is to slicing the proofs (192 bytes) to several 32 bytes chunks and encoded in XCM multi-location. It is a kind of hacky way of using XCM multi-location but this would not introduce storage bloat-up per se.

[KiChjang]

Reposting what Gav said in a channel:

You are misusing MultiLocation/Junction.

MultiLocation is a type to identify a single data-structure element held in consensus.

Junction identifies one based on a basic transformation of another.

Instead you should refer to the location in consensus directly, not via an indirect (and thus longer) description.

one simple way would be to hash the concatenated pairs.

then look it up in a map in your registry.

But in your case, you'd hash the concatenated proofs, rather than the concatenated pairs.

[stechu]

Reposting what Gav said in a channel:

You are misusing MultiLocation/Junction.

MultiLocation is a type to identify a single data-structure element held in consensus.

Junction identifies one based on a basic transformation of another.

Instead you should refer to the location in consensus directly, not via an indirect (and thus longer) description.

one simple way would be to hash the concatenated pairs.

then look it up in a map in your registry.

But in your case, you'd hash the concatenated proofs, rather than the concatenated pairs.

Agreed that multi-location is not a very ideal place for putting ZKP for transactions. I guess the general gaol that we are trying to avoid out-of-band messaging here. This complicate the dApp implementations. Maybe the "right" way is to use transact directly.