Pull Request Summary
Add the experimental XCM CE along with companion pallet.
Callback Design
The callback design make use of pallet_xcm's OnResponse handler which has capability to notify a dispatch on a XCM response (if notify query is registered).
For us that dispatch is companion pallet's on_callback_received which will route the xcm response (Response enum) back to contract via a bare_call (if wasm contract)
Let's understand the control flow,
Contract A will call the XCM CE (XCMExtension) to register a new query of type QueryType::WasmContractCallback { .. } .
CE/Precompile’s will in-turn call XCM Transact’s new_query
Register the query with help of pallet_xcm's new_notify_query and sets the notify’s dispatch to its dispatchable on_callback_recieved() and saves the QueryType for the registered query_id and return the query_id.
Contract A send the XCM with help of CE/Precompile
CE/Precompile forward the XCM to XCM Transact
XCM Transact forward the XCM to Pallet Xcm
which finally sends it to destination
Destination sends the response back which is intercepted by pallet_xcm ’s OnResponse
pallet_xcm’s OnResponse calls the on_callback_recieved()
on_callback_recieved() on receiving response calls the CallbackHandler
CallbackHandler will then route the response based on the QueryType , for example to a Wasm contract’s method
Open Questions
1. Who pays for the execution of callback?
In current design, when registering a query callback weight is not taken into account. This is because the weight of callback/notify is set inside the query instruction itself and query method does not take xcm as input (chicken and egg problem), for example in the below instruction.
ReportTransactStatus(QueryResponseInfo {
destination: (Parent, Parachain(1)).into(),
query_id,
max_weight: Weight::from_parts(100_000_000_000_000, 1024 * 1024 * 1024)), -----> this will be copied to `QueryResponse` that will be sent to querier chain
})
For the above query instruction below query response would be sent and max_weight is copied to it. The OnResponse handler will ensure that notify dispatch's weight is less than max_weight before calling it.
via PriceForSiblingDelivery - When calculating the XCM sending fee, this is ideal place for it actually. The XCM sending fee should take into account the max_weight from every query instruction. Right now, PriceForSiblingDelivery in xcm router is set to ().
3. What happens when callback failed?
If in any situation callback fails due to any reason (gas limit, wasm trapped, etc) that would quite difficult for contract to manage. It's still not fatal because contract can implement it's own query timeout based on block number and consider the query expired in such case. Having said that, it would be a very good idea to save the response for contract to manually poll later as a fallback in case of callback error.
Obviously rent fee + weight associated with it should be charged during query registration itself which will make registering query a bit costlier.
Still a TODO
4. How is rent fee handled for registering query and rent refund?
Not handled right now, still a TODO
5. Query Ids cannot grow indefinitely
We are using pallet_xcm's query mechanism for this registering/managing queries which does not recycle query id, that means query id is incremental, thus limited. On top of that, pallet_xcm uses u64 for query ids, which although has very large max value but still can be a issue in future (well if that comes, we'll probably need a storage migration to change it to u128).
TODO
[x] move test scenarios to this PR and add more (need xcm simulator?)
[x] figure out overall weights (new query, take response, etc) via benchmarks
Pull Request Summary Add the experimental XCM CE along with companion pallet.
Callback Design
The callback design make use of
pallet_xcm
'sOnResponse
handler which has capability to notify a dispatch on a XCM response (if notify query is registered).For us that dispatch is companion pallet's
on_callback_received
which will route the xcm response (Response
enum) back to contract via abare_call
(if wasm contract)Let's understand the control flow,
Contract A
will call the XCM CE (XCMExtension
) to register a new query of typeQueryType::WasmContractCallback { .. }
.new_query
pallet_xcm
'snew_notify_query
and sets the notify’s dispatch to its dispatchableon_callback_recieved()
and saves theQueryType
for the registeredquery_id
and return thequery_id
.Contract A
send the XCM with help of CE/Precompilepallet_xcm
’sOnResponse
pallet_xcm
’sOnResponse
calls theon_callback_recieved()
on_callback_recieved()
on receiving response calls theCallbackHandler
CallbackHandler
will then route the response based on theQueryType
, for example to a Wasm contract’s methodOpen Questions
1. Who pays for the execution of callback?
In current design, when registering a query callback weight is not taken into account. This is because the weight of callback/notify is set inside the query instruction itself and query method does not take xcm as input (chicken and egg problem), for example in the below instruction.
For the above query instruction below query response would be sent and
max_weight
is copied to it. TheOnResponse
handler will ensure that notify dispatch's weight is less thanmax_weight
before calling it.Therefore, the weight for callback and for processing the callback should be charged when when sending the XCM. This can be achieved in two manners,
WeightInfoBounds
: where all query instructions'sQueryResponseInfo::max_weight
is taken into account. Since weight ofpallet_xcm::send()
dispatch also includes the XCM weight. Kusama uses WeightInfoBounds but does not takemax_weight
into account, it would be similar but also takemax_weight
into account.PriceForSiblingDelivery
- When calculating the XCM sending fee, this is ideal place for it actually. The XCM sending fee should take into account themax_weight
from every query instruction. Right now,PriceForSiblingDelivery
in xcm router is set to()
.3. What happens when callback failed?
If in any situation callback fails due to any reason (gas limit, wasm trapped, etc) that would quite difficult for contract to manage. It's still not fatal because contract can implement it's own query timeout based on block number and consider the query expired in such case. Having said that, it would be a very good idea to save the response for contract to manually poll later as a fallback in case of callback error. Obviously rent fee + weight associated with it should be charged during query registration itself which will make registering query a bit costlier. Still a TODO
4. How is rent fee handled for registering query and rent refund?
Not handled right now, still a TODO
5. Query Ids cannot grow indefinitely
We are using
pallet_xcm
's query mechanism for this registering/managing queries which does not recycle query id, that means query id is incremental, thus limited. On top of that,pallet_xcm
usesu64
for query ids, which although has very large max value but still can be a issue in future (well if that comes, we'll probably need a storage migration to change it tou128
).TODO
Check list