polkascan / py-substrate-interface

Python Substrate Interface
https://polkascan.github.io/py-substrate-interface/
Apache License 2.0
240 stars 114 forks source link

Question: What is the syntax for the nested parameters of an XCM call? #262

Closed arseniiarsenii closed 1 year ago

arseniiarsenii commented 1 year ago

Hello. I am trying to send an XCM call with this library to place a pin order in Crust Shadow parachain using an XCM channel established between it and Robonomics.

From what I see in the code snippets in the docs, the names of the pallets, extrinsics and call parameters accepted by this library are different from the ones, displayed in the polkadot.js dapp in the developers/extrinsics section. For instance, pallets in Substrate-interface follow pythonic PascalCase naming convention as opposed to camelCase displayed in the dapp. Extrinsics and their parameters use snake_case instead of camelCase as if they were python methods and variables.

However, I didn't find any mention of this syntax difference in the documentation. This raises a question for me: do I have to change all the pallet, extrinsic, and parameter names in my XCM call, which is a quite deeply nested structure? Will they all be later parsed recursively and converted to the values, accepted by the node? Is it even necessary to change their names, or is this just a convenience feature for the developers used to python naming conventions?

I have composed an XCM call using the polkadot.js dapp which looks something like this when decoded:

dest: XcmVersionedMultiLocation

{
  V1: {
    parents: 1
    interior: {
      X1: {
        Parachain: 2,012
      }
    }
  }
}

message: XcmVersionedXcm

{
  V2: [
    {
      WithdrawAsset: [
        {
          id: {
            Concrete: {
              parents: 0
              interior: Here
            }
          }
          fun: {
            Fungible: 1,000,000,000,000
          }
        }
      ]
    }
    {
      BuyExecution: {
        fees: {
          id: {
            Concrete: {
              parents: 0
              interior: Here
            }
          }
          fun: {
            Fungible: 1,000,000,000,000
          }
        }
        weightLimit: Unlimited
      }
    }
    {
      Transact: {
        originType: Native
        requireWeightAtMost: 1,000,000,000
        call: {
          encoded: 0x7f00b8516d54507739795a7265397a68773458704139613738344754713878576b535a63586a7a38777a314c4635466e63b80b000000000000
        }
      }
    }
    RefundSurplus
    {
      DepositAsset: {
        assets: {
          Wild: All
        }
        maxAssets: 1
        beneficiary: {
          parents: 0
          interior: {
            X1: {
              Parachain: 2,048
            }
          }
        }
      }
    }
  ]
}

From my understanding, I should provide the following python dictionary to the parameters of the extrinsic call in code:

{
            "dest": {
                "V1": {
                    "parents": 1,
                    "interior": {
                        "X1": {
                            "Parachain": 2012,
                        }
                    },
                }
            },
            "message": {
                "V2": [
                    {
                        "WithdrawAsset": [
                            {
                                "id": {"Concrete": {"parents": 0, "interior": "Here"}},
                                "fun": {"Fungible": 10**12},
                            }
                        ]
                    },
                    {
                        "BuyExecution": {
                            "fees": {
                                "id": {"Concrete": {"parents": 0, "interior": "Here"}},
                                "fun": {
                                    "Fungible": 10**12,
                                },
                            },
                            "weightLimit": "Unlimited",
                        }
                    },
                    {
                        "Transact": {
                            "originType": "Native",
                            "requireWeightAtMost": 10**9,
                            "call": {
                                "encoded": crust_encoded_call_data,
                            },
                        }
                    },
                    "RefundSurplus",
                    {
                        "DepositAsset": {
                            "assets": {"Wild": "All"},
                            "maxAssets": 1,
                            "beneficiary": {
                                "parents": 0,
                                "interior": {"X1": {"Parachain": 2048}},
                            },
                        }
                    },
                ]
            },
        }

Is this good to use as it is, or am I doing something wrong from the syntax standpoint? Thank you in advance.

arjanz commented 1 year ago

I'm working on a function that will give the necessary insight for these complex calls, which I will release in a few days. In the meantime what can help is to retrieve an existing extrinsic containing the call to figure out how to format the call args:

receipt = substrate.retrieve_extrinsic_by_identifier("12747310-2")
print(receipt.extrinsic['call']['call_args'])
arseniiarsenii commented 1 year ago

This is a nice touch. Still, I think it's worth documenting this formatting principles in the library documentation. Thank you for your response.

arjanz commented 1 year ago

Yes indeed, the docs are lacking the information about how to encode Enum, Struct and other primitives, I'll make an issue for that.

The remaining problem though is the complexity of certain calls or storage paramaters and the generic nature of Substrate (lots of different implementations, runtime upgrades and changing specifications of call args), better helper functions will save the developer a lot of time. I'll share with you when I have a working version.

Were you able to figure out the correct call argument format?

arjanz commented 1 year ago

This is the output for the XcmPallet.execute call, hopes this helps: https://gist.github.com/arjanz/10b6262384b875b03bb4ab8fd59db3db

arjanz commented 1 year ago

One big difference between the Python and PolkadotJS implementation is that we chose to maintain the original snake case of the parameters and variables as they are in Substrate (which conveniently is also the convention in Python) , where PolkadotJS converts them to CamelCase, which is more common for JS.

arjanz commented 1 year ago

Your feedback has been processed in the latest release, there is a helper function available to have more insight of type decomposition https://github.com/polkascan/py-substrate-interface#type-decomposition-of-call-params

Also some encoding principles are documented: https://github.com/polkascan/py-scale-codec#examples-of-different-types

arjanz commented 1 year ago

@arseniiarsenii Latest addition, this might also come in handy: https://polkascan.github.io/py-substrate-metadata-docs/