Testnet faucet program - design discussion

[ameba23]

As discussed on discord there is a plan to have a testnet faucet account, where people who want testnet tokens can request to sign a balance transfer from the faucet account to an account of their choosing.

The program for the testnet faucet account should ideally check that the payload is decodes to a balance transfer call and that the amount transferred is below some limit.

This sounds easy, but i think doing it will require the program having access to the entropy runtime configuration in order to decode the extrinsic, as well as the subxt crate, which might make the program binary quite big. I'm also not totally sure how to decode a subxt::tx::PartialExtrinsic<EntropyConfig, OnlineClient<EntropyConfig>>, but it must be doable.

As for how to actually create sign and submit the extrinsics, this is how i would go about it in rust - but the plan is to do this in JS. Hopefully polkadot-js provides and easier way of adding a signature to a partial extrinsic.

async fn create_payload_to_sign(
    api: &OnlineClient<EntropyConfig>,
    rpc: &LegacyRpcMethods<EntropyConfig>,    
    to: SubxtAccountId32,
) -> anyhow::Result<PartialExtrinsic<EntropyConfig, OnlineClient<EntropyConfig>>> {
    let call = entropy::tx().balances().transfer_allow_death(MultiAddress::Id(to), FAUCET_AMOUNT);

    let block_hash = rpc.chain_get_block_hash(None).await?.ok_or(SubstrateError::BlockHash)?;
    let nonce_call =
         entropy::apis().account_nonce_api().account_nonce(TESTNET_FAUCET_ACCOUNT_ID.clone());
    let nonce = api.runtime_api().at(block_hash).call(nonce_call).await?
    let latest_block = api.blocks().at_latest().await?;
    let tx_params =
        Params::new().mortal(latest_block.header(), MORTALITY_BLOCKS).nonce(nonce.into()).build();

   api.tx().create_partial_signed(call, TESTNET_FAUCET_ACCOUNT_ID, tx_params)?
}

Then we would make an entropy signature request with partial_extrinsic.signer_payload() as the message and submit to entropy and get a 65 byte signature.

Then we can add the signature using PartialExtrinsic::sign_with_address_and_signature:

let submittable_extrinsic = partial_extrinsic.sign_with_address_and_signature(
    TESTNET_FAUCET_ACCOUNT_ID.into(),
    subxt::MultiSignature::Ecdsa(signature_from_entropy),
);
submitable_extrinsic.submit_and_watch().await?;

[ameba23]

Update - i started trying to write a program for this and i cannot get subxt to compile with cargo component:

Cargo.toml:

[dependencies]
entropy-programs-core = { workspace = true }
subxt = { version = "0.35.3", default-features=false, features=["web"] }

Trying to compile gives this error:

cargo component build --release --target wasm32-unknown-unknown
   Encoding target for template-barebones (/home/turnip/radish/src/entropy/programs/target/bindings/template-barebones/target.wasm)
   Compiling template-barebones v0.1.0 (/home/turnip/radish/src/entropy/programs/examples/barebones)
    Finished `release` profile [optimized] target(s) in 0.51s
    Creating component /home/turnip/radish/src/entropy/programs/target/wasm32-unknown-unknown/release/template_barebones.wasm
error: module requires an import interface named `__wbindgen_placeholder__`

I am looking into this but i don't know cargo-component so well. Compiling on wasm without cargo-component works fine.

[ameba23]

@JesseAbram suggested using substrate-api-client instead of subxt.

I can confirm that substrate-api-client does compile in a program when default features are disabled.

But it does not appear to have the PartialExtrinsic type that we need to decode with SCALE.

Although i am wondering now - what parts of a transaction actually get signed? Looking at the source code of PartialExtrinsic it looks like it is 'call data' plus ExtrinsicParams: https://docs.rs/subxt/latest/src/subxt/tx/tx_client.rs.html#344

ExtrinsicParams, is (generic) in the low level ac-primitives crate: https://docs.rs/ac-primitives/0.9.1/ac_primitives/extrinsics/extrinsic_params/trait.ExtrinsicParams.html

I have the feeling there must be an easier way to decode an extrinsic payload, but this should work.

[ameba23]

I tried writing a test that would decode a partial balance transfer extrinsic (ignoring problems with compiling things for cargo-component for now).

Here the extrinsic is created: https://github.com/entropyxyz/entropy-core/blob/46a36cf0ef9504141a617b876b8edc09778f1bb2/crates/client/src/client.rs#L378

Here we attempt to decode it (this is what the program would need to do):

https://github.com/entropyxyz/entropy-core/blob/46a36cf0ef9504141a617b876b8edc09778f1bb2/crates/client/src/client.rs#L391

here is a test which does both - but fails as the call data also contains some chain-specific metadata: https://github.com/entropyxyz/entropy-core/blob/46a36cf0ef9504141a617b876b8edc09778f1bb2/crates/threshold-signature-server/tests/sign.rs#L41

[ameba23]

Had a look at this today together with @HCastano and made some progress towards being able to decode a balance transfer call data. This isn't the full signed payload, but the first part of it.

The issue was we were attempting to decode the entropy::balances::Call enum but the actual type of the call is entropy::balances::calls::types::TransferAllowDeath.

However, when we run the test we get a different error:

thread 'integration_test_sign_partial' panicked at crates/client/src/client.rs:399:90:
called `Result::unwrap()` on an `Err` value: Error { cause: Some(Error { cause: None, desc: "Could not decode `MultiAddress`, variant doesn't exist" }), desc: "Could not decode `TransferAllowDeath::dest`" }

There is an issue with how we are using subxt::utils::MultiAddress to represent the to field of the transfer. I've tried both the ::Id and ::Address32 variants and get the same error. Its as if it is expecting a different type.

Here is the latest version of the create_partial_balance_tx and decode_partial_balance_tx functions: https://github.com/entropyxyz/entropy-core/blob/ee92f1259a2dede5aa2c580d3414f43f188a16fe/crates/client/src/client.rs#L394

[HCastano]

I've simplified the reproduction setup a bit.

Basically I made a new crate and the only thing inside is this:

#[subxt::subxt(
    runtime_metadata_path = "/Users/hcastano/entropy-core/crates/client/entropy_metadata.scale",
    substitute_type(
        path = "entropy_shared::types::KeyVisibility",
        with = "::subxt::utils::Static<::entropy_shared::KeyVisibility>",
    ),
    substitute_type(
        path = "entropy_shared::types::ValidatorInfo",
        with = "::subxt::utils::Static<::entropy_shared::ValidatorInfo>",
    )
)]
pub mod entropy {}

use parity_scale_codec::Decode;
use entropy::balances::calls::types::TransferAllowDeath;

fn main() {
    let transfer_allow_death_encoded_call = "0x49028400d43593c715fdd31c61141abd04a99fd6822c8558854ccde39a5684e7a56da27d014477216a9478f6d837f456f6a57d09404cf3e90609cb89d183aa7de4375571089dfca9c3bbeb1a3af0ac6d45c17eccb75934b423ac6913e01dbf040d8440878fa50000000700008eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a480f0080c6a47e8d03";
    let mut input = transfer_allow_death_encoded_call.as_ref();

    TransferAllowDeath::decode(&mut input).unwrap();
}

I got the hex string from a manual transfer I did on a local network (e.g --dev).

I get the same MultiAddress decoding error as above.

However, if you notice @ameba23 the extrinsic produced by PJS Apps does differ from the one in your test. Just something to keep in mind going forward.

[ameba23]

Ah great this should make testing this much simpler

However, if you notice @ameba23 the extrinsic produced by PJS Apps does differ from the one in your test. Just something to keep in mind going forward.

Strange that its so much longer - 148 bytes, whereas i was getting 40 bytes for just the call data and 116 bytes for the full payload to be signed. Are you sure this is not an already signed extrinsic?

[ameba23]

Still stuck on this error when attempting to decode a balance transfer call: "Could not decode MultiAddress, variant doesn't exist"

I noticed there is both a sp_runtime::MultiAddress and a subxt::utils::MultiAddress. To avoid any ambiguity i tried specifying the exact type which our balance call uses (which i think is just a type alias to subxt::utils::MultiAddress):

    let to: <EntropyConfig as Config>::AccountId = to.into();
    let to_multi = entropy::balances::calls::types::transfer_allow_death::Dest::Id(to);
    let call = entropy::tx().balances().transfer_keep_alive(to_multi, amount);

But this still gives the same error when trying to decode. I don't know what else to try.

[JesseAbram]

So I looked into this today and my suggestions are for a "generalized substrate program" (as in we can build this and then add the faucet logic in later

    let unsigned_extrinsic = offline_api.tx().create_partial_signed_offline(&substrate_tx, tx_params).unwrap().signer_payload();

using partial tx and an offline api (in the program at least we will need this because of sandboxing) a user can create the message to be signed

To get the payload they pass through the data needed for the tx (the params, nonce, call etc) in the aux data with the info needed to create an offline_api (metadata, genesis hash etc)

In the program we create the partial and get the signed_payload then compare that signed payload and make sure it matches the message signed payload

Since they are the same we can now use the info from the payload to match any constraints we want and the original message will get passed to synedrion to sign

[ameba23]

Ah you mean actually re-create the transaction inside the program, now i get it. Yep that is a very good idea.

But its going to be more needy in terms of dependencies. I can't get subxt to compile to a webassembly component (even though it will compile to browser wasm), so i was imagining we use some of its more low level dependencies to do this. The tricky bit is our chain config - i don't know how we can get that without subxt.

I still really feel like it should be possible to decode the call data from the signer payload itself as it is only hashed if it over 256 bytes: https://docs.rs/subxt-core/latest/src/subxt_core/tx/mod.rs.html#197

[ameba23]

Ok so as @JesseAbram figured out, subxt actually will compile to a webassembly component just fine with the native feature enabled. I didn't bother trying, just assumed for sure it wouldnt work.

[ameba23]

So @JesseAbram has written (or almost finished) the program: https://github.com/entropyxyz/faucet_program

As for signing a PartialExtrinsic with the signature produced by entropy, i am guessing it will be much the same as if we made one with the k256 crate. I tried to do that today and this is what i have:

The bit that took me a while to figure out was that to get an account ID from a 33 byte ecdsa public key we have to hash it. See: https://substrate.stackexchange.com/questions/8158/how-to-properly-use-accountid-for-ecdsa-keypairs

use entropy_client::chain_api::{entropy, get_api, get_rpc, EntropyConfig};
use sp_io::hashing::blake2_256;
use subxt::{
    config::{substrate::MultiSignature, PolkadotExtrinsicParamsBuilder as Params},
    ext::sp_core::{sr25519, Pair},
    tx::PairSigner,
    utils::{AccountId32, MultiAddress},
    Config,
};

use k256::ecdsa::{signature::Signer, Signature, SigningKey, VerifyingKey};
use rand_core::OsRng;

#[tokio::main]
async fn main() {
    // A k256 signing key - this would be made with entropy DKG
    let signing_key = SigningKey::random(&mut OsRng);

    // The verifying key from entropy
    let verifying_key = VerifyingKey::from(&signing_key);

    let verifying_key_bytes: [u8; 33] = verifying_key
        .to_encoded_point(true)
        .as_bytes()
        .to_vec()
        .try_into()
        .unwrap();

    // Hash it to get account id:
    let account_id = AccountId32(blake2_256(&verifying_key_bytes));

    let endpoint_addr =
        std::env::var("ENTROPY_DEVNET").unwrap_or("ws://localhost:9944".to_string());
    let api = get_api(&endpoint_addr).await.unwrap();
    let rpc = get_rpc(&endpoint_addr).await.unwrap();

    // Use alice as the 'to' address:
    let to: <EntropyConfig as Config>::AccountId = {
        let p_alice = <sr25519::Pair as Pair>::from_string("//Alice", None).unwrap();
        let signer_alice = PairSigner::<EntropyConfig, sr25519::Pair>::new(p_alice);
        let account_id = signer_alice.account_id();
        account_id.clone().into()
    };

    // Make a partial extrinsic
    let partial_extrinsic = {
        let call = entropy::tx()
            .balances()
            .transfer_allow_death(subxt::utils::MultiAddress::Id(to), 1_000_000);

        let block_hash = rpc.chain_get_block_hash(None).await.unwrap().unwrap();

        let nonce_call = entropy::apis()
            .account_nonce_api()
            .account_nonce(account_id.clone());

        let nonce = api
            .runtime_api()
            .at(block_hash)
            .call(nonce_call)
            .await
            .unwrap();

        let latest_block = api.blocks().at_latest().await.unwrap();
        let tx_params = Params::new()
            .mortal(latest_block.header(), 100u64)
            .nonce(nonce.into())
            .build();

        api.tx()
            .create_partial_signed(&call, &account_id, tx_params)
            .await
            .unwrap()
    };

    // Sign it - here with k256 but it should work just the same with entropy
    let signature: Signature = signing_key.sign(&partial_extrinsic.signer_payload());
    let submittable_extrinsic = partial_extrinsic.sign_with_address_and_signature(
        &MultiAddress::Id(account_id),
        &MultiSignature::Ecdsa(signature.to_vec().try_into().unwrap()),
    );

    // Sumbit as usual
    submittable_extrinsic
        .submit_and_watch()
        .await
        .unwrap()
        .wait_for_finalized()
        .await
        .unwrap();
}