Pre-RFC: XMS language (XCM Made Simple)

[olanod]

Context: This is an early stage idea to gather some feedback, it's a byproduct of the design of "SubeX", the next version of Sube that aims to be XCM centric(a simple/lightweight client that can(will) run anywhere).

---

XMS is(would be) a KDL-based client agnostic format/language for representing XCM instructions and extrinsics, it's a text based document language with xml-like semantics to be machine and human readable. With XMS one can declare snippets of code that abstract low level XCM instructions to later compose them creating parametrisable scripts that are simple to read and write.

idea of an XMS script adapted from this Moonbeam example

import "./prelude.xms" // import type declarations from other files/URLs

const "candidate" val="0x1234"

// Define custom calls that return an assembled extrinsic
// it can define custom arguments or accept children that are injected in a designated "slot"
cmd "stake" amount=null {
    call pallet="parachain_staking" x="delegate_with_auto_compound" {
        candidate="$candidate"
        amount="$amount"
        auto_compound=100
        // We can regiter extensions to evaluate values with custom types
        candidate_delegation_count=(query)"parachain_staking/candidate_info/$candidate | get delegation_count"
        candidate_auto_compounding_delegation_count=(query)"parachain_staking/auto_compounding_delegations/$candidate | len"
    }
}

// Declare a series of XCM instructions that can be parametrized with arguments 
// or extended children nodes that are injected in a "slot"
cmd "remote_stake" amount=null {
    xcm
        withdraw_asset {
             asset id="./pallet_instance(3)" fun=100_000_000_000_000_000
        }
        buy_execution fees="./pallet_instance(3)/$100_000_000_000_000_000" weight_limit="unlimited"
        transact \
            origin_kind="sovereign_account" \
            require_weight_at_most="(40_000_000_000,900_000)" \
            call=(cmd)"stake $amount | encode"
}

// scripts should call one or more definitions
remote_stake 500_e10

Basing XMS in KDL makes its implementation easy and client/language agnostic(libraries in multiple languages). It's a language with great syntax that makes it feel like you designed a specific purpose language but at the end of the day it's just an XML-like document with nodes, arguments and children, tools and clients can "expand" an XMS document into a concrete extrinsic or XCM ready to be signed/submitted or generate code for clients or pallets.

A side effect of working on this format is having well defined string representations of common types like MultiLocation

I believe having a simple way to declare composable snippets of XCM is the best way to abstract it, maintaining libraries for specific languages with a limited amount of commonly used patterns doesn't scale well. Instead letting the community come up with scripts that can be shared around can allow for better experimentation until eventually we land into a stable collections of "preludes" suitable for different use cases.

[bkchr]

I'm all into making XCM better approachable. However, I don't think that using XML is that much of a better idea here. Maybe just having nice abstractions in different languages for building XCM programs is the better way for now.

[kianenigma]

I would be curious to see the Rust equivalent of the program you have above, and a couple of other programs, and demonstrate how this will simplify it? First glance, I feel like it would be more or less equally complicated.

[shawntabrizi]

What would be the action item for this RFC? How would this ultimately effect the polkadot-sdk code base?

[xlc]

If the goal is to construct XCM from more programming languages, an alternative approach will be SDK generation from the Rust defs. As we have type metadata available, it is not hard to generate SDK for other languages such as JS/Python/Solidity, or even XCM/JSON schema.

[olanod]

If the goal is to construct XCM from more programming languages, an alternative approach will be SDK generation from the Rust defs. As we have type metadata available, it is not hard to generate SDK for other languages such as JS/Python/Solidity, or even XCM/JSON schema.

Yeah one of the main goals is to construct XCM and complex extrinsics from more programming languages in a simple declarative way as XCM instructions lend themselves pretty well to be written that way instead of an imperative approach(Alberto didn't enjoy making the Moonbeam JS examples), specially being able to write the instructions from the many client-side languages that interact with the chains. Sube a very lightweight/simple Rust client designed for embedded devices would be the first XMS "interpreter", it currently has experimental JS bindings with planned support for Python/Kotlin/Swift.

It's also a goal to steer away from writing XCM with Rust. we should pivot it to focus things on the client side, most developers are NOT Rust developers and they are the ones that can bring the use cases for the format. With pallet XCM's send/execute we have most of the tools we need on the chain side, instead of writing more wrapper pallets I'd like to see a vibrant ecosystem of (client-side)developers sharing XCM abstractions in a language agnostic way. The idea of generating SDKs form Rust code is good and a step forward over having to replicate the same work in different languages but I think we(Rust devs) don't know yet what abstractions we need, let the users figure that out after lots of easy experimentation.

import xms from 'https://dot.tools/xms.js'
// imagining a curious JS user importing some well known script
await xms.import("https://acala.dev/xcm/x-tokens.xms")
// she can experiment with ad-hoc scripts
await xms.load(xms`
import 'https://dot.tools/xcm/std.xms'
def "me" "0x1234567890"
def "half-my-money" {
    query "/polkadot/system/account/{me} | get data.free | {in} / 2"
}
`)

await xms('half-my-money | x-transfer /polkadot/${in} ./:2000')

[olanod]

I would be curious to see the Rust equivalent of the program you have above, and a couple of other programs, and demonstrate how this will simplify it? First glance, I feel like it would be more or less equally complicated.

The JS code in Moonbeam's example is already more complicated I can imagine the Rust code won't be much different?

What would be the action item for this RFC? How would this ultimately effect the polkadot-sdk code base?

I'll work a bit more on the ergonomics of the format, decide on a minimal set of "reserved keywords"(e.g. a speciall def node used to construct everything else?) and try to come up with a "standard library" that would just be common definitions that expand into the usual verbose JSON-like representation we usually use when serializing Rust types. A PoC with working examples would also be good for getting a feeling about the language, based on the feedback during the call, I'd consider the experiment failed if it doesn't prove to be simple enough to use and learn.
I'm not sure If I understand the polkadot-sdk concern part, for now it doesn't have any impact? eventually we could add the xms crate in the repo and have it come with tools like the interpreter, code generators or keep the "official" collection of xms scripts that can be published on IPFS or a static web server. It can also be kept independent just like polkadot.js or smoldot are not part of the official repo but still fall under the umbrella of the fellowship.

[shawntabrizi]

I guess the reason why I ask about the impact on the Polkadot SDK is due to what I envision these RFCs to be about.

And I could totally be wrong here, so happy to have a more meta conversation and realign my thinking, but...

It seems this repo, and the Polkadot fellowship are about making decisions around the core protocols of Polkadot and the Polkadot SDK.

Your initiative and goals here are cool, but seem to be something external from Polkadot and the Polkadot SDK. For example, the creation of XMS would not prohibit the creation of any number of other external declarative languages to wrap XCM. There is no reason that I see so far why this meta-language needs to be enshrined into a Polkadot RFC.

Since this proposal does not actually seem to have any effect to the Polkadot Protocol or the Polkadot SDK (which contains the XCM crates), then there really isn't any need for this fellowship to approve or deny this RFC.

It seems instead, this is just a way to communicate ideas and get feedback, which is great in its own right, but probably not the intention of RFCs process.

Does this resonate with anyone else? Am I in the wrong mindset?

Perhaps we need a separate repo, or section in the Polkadot Forum, which is more like "talk with and get feedback from the fellows".

---

All this being said, I do resonate with your goals, and I think that the experimentation of things like this is valuable and should be supported.

[KiChjang]

To write XCM and send it over the wire, all you need is to have a SCALE codec in your programming language, and create structs/classes/types that have the same SCALE encoding. An entirely new language is not required at all.

In fact, this really sounds like it should be an OpenGov proposal to develop SCALE codec libraries and primitive XCM types for other programming languages. I struggle to find any sort of action item for the fellowship here in this proposal.