Inquiry regarding scope for proposed Polkadot Provider API RFC

[josepot]

Hello Polkadot Fellows,

I am currently spearheading an initiative aimed at significantly enhancing the Polkadot ecosystem's infrastructure. Before I proceed with a detailed RFC submission, I seek your guidance to ensure that my proposal aligns with the scope of this repository.

Proposal Overview

The project that I'm currently working on involves developing a light-client first library, envisioned as a successor to PolkadotJS. A central aspect of this initiative is the creation of a standardized Polkadot Provider API. This API is designed to be library agnostic, setting a foundation for a diverse ecosystem of tools and applications.

Key Objectives

• Establish a standardized, library-agnostic API for dApps to interact seamlessly with the Polkadot network.
• Foster an ecosystem where various tools and applications can be developed around this standardized API.
• Enhance interoperability and developer experience within the Polkadot ecosystem.

Specific Inquiries

1. Language Specificity: The initial proposal is likely to be JavaScript/TypeScript-centric. However, I'm open to considering a more generic approach, possibly leveraging a JSON-RPC based API, to ensure broader applicability and inclusiveness. Would a language-specific proposal be suitable for this repository? If not, would a shift towards a more generic, protocol-based approach be more appropriate?

2. Fit within the Current Scope: One of the key aspects of my proposal is the establishment of a simplified protocol that enables dApps to interact seamlessly with both the JSON-RPC endpoints provided by a node, and a set of functions (or JSON-RPC calls) designed for interfacing with keyring agents present in various environments. This dual interaction model aims to streamline and standardize the way dApps communicate with the Polkadot network.

However, it is not entirely clear to me whether this proposal, focusing on a standardized protocol for a node and keyring interactions, falls within the current scope of this repository. The proposal extends beyond mere feature enhancement, aiming to introduce a fundamental protocol layer that could serve as a cornerstone for future development within the Polkadot ecosystem. As such, I am seeking clarification on whether the scope of this repository encompasses the establishment of such foundational protocols.

Thank you for your consideration and I look forward to your valuable feedback.

Best regards,

Josep

[tomaka]

I don't think this belongs here.

The reason why we have RFCs for changes to the runtime and the client specification is that there can fundamentally only be one runtime and one client specification. You can't have for example one client implementation use a protocol and a different client implementation use another, that won't work. For this reason, every change to the protocol go through an RFC so that everyone can weight in.

When it comes to a UI library API, that's not true. We can completely have multiple different APIs in parallel, and multiple different versions of an API in parallel.

---

Aside from the question of whether it belongs to the RFCs repo, my opinion is also that the idea of having a "standard" API is by itself a bad idea. You're just intentionally making your job more difficult for, in practice, no benefit.

People will just build tools on top of your library, there's no need for any standard to exist as long as there isn't another library that needs to be compatible with yours. And what would be the point of another library that has to conform to the exact same API? How could you implement the exact same API in two different ways without making the API leaky and thus bad? Any meaningful difference between libraries requires different APIs.

I would personally recommend you to put this "standard API" idea on the side and revisit it later if there's an actual very pragmatic (not just theoretical) advantage in having one.

[josepot]

Hi @tomaka ! Thanks for your insights, greatly appreciated! 🙏

Just to be clear, the RFC that I would like to submit would take a lot of inspiration from the EIP-1193, but for Polkadot.

I would personally recommend you to put this "standard API" idea on the side and revisit it later if there's an actual very pragmatic (not just theoretical) advantage in having one.

That's actually what I have been doing so far. However, yesterday I had a conversation with @jsdw and he also saw the value of defining this Provider, because that would greatly simplify the integration of dApps build with Subxt. So, I think that there is already a pragmatic advantage.

[josepot]

To further clarify, the goal of this RPC would be to "standarize" the following interfaces, plus some known errors, etc. This is by no means a formal proposal, this is just so that you get an idea of what it is that we would like to standarize. Also, please notice that the discoverability of Providers is out of scope for this proposal.

type Callback<T> = (value: T) => void
type UnsubscribeFn = () => void

export interface Chain {
  chainId: string
  name: string
  symbol: string
  decimals: number
  ss58Format: number

  // it pulls the current list of available accounts for this Chain
  getAccounts: () => Promise<Array<Account>>

  // registers a callback that will be invoked whenever the list
  // of available accounts for this chain has changed. The callback
  // will be synchronously called with the current list of accounts.
  onAccountsChange: (accounts: Callback<Array<Account>>) => UnsubscribeFn

  // returns a JSON RPC Provider that it's compliant with new
  // JSON-RPC API spec:
  // https://paritytech.github.io/json-rpc-interface-spec/api.html
  connect: (
    // the listener callback that the JsonRpcProvider
    // will be sending messages to
    onMessage: Callback<string>,
  ) => JsonRpcProvider
}

export interface JsonRpcProvider {
  // it sends messages to the JSON RPC Server
  send: (message: string) => void

  // `publicKey`: is the public key of the signer account
  // `callData`: is the scale encoded call-data
  // (module index, call index and args)
  // `hintSignedExtensions`: is an **optional** argument which
  // allows the dApp to suggest certain values for some signed-extensions.
  // However, its worth noting that the Provider may decide to use other
  // values based on user input collected after presenting the UI with the
  // decoded data to the user.
  createTx: (
    publicKey: Uint8Array,
    callData: Uint8Array,
    hintSignedExtensions?: Record<string, Uint8Array>,
  ) => Promise<Uint8Array>

  // `publicKey`: is the public key of the signer account
  // `message`: the data that the user will be requested to sign
  signData: (
    publicKey: Uint8Array,
    message: Uint8Array
  ) => Promise<Uint8Array>

  // it disconnects from the JSON RPC Server and it de-registers
  // the `onMessage` and `onStatusChange` callbacks that
  // were previously registered
  disconnect: UnsubscribeFn
}

export interface Account {
  // public key of the account
  publicKey: Uint8Array

  // The provider may have captured a display name
  displayName?: string
}

[tomaka]

the RFC that I would like to submit would take a lot of inspiration from the EIP-1193, but for Polkadot.

Just because Ethereum did that doesn't mean that it's a good idea.

the goal of this RPC would be to "standarize" the following interfaces

But what's the point of "standardizing" that?

Can PolkadotApi and Subxt not just both implement a similar API?

A dApp would only ever use either PolkadotApi or Subxt. They choose between the two which provider they want to use. Having a standard API would help in one very specific situation: if a dApp wants to migrate between PolkadotApi and Subxt, and that the APIs are not exactly the same, they would need to adjust their code. Do you really think that this is such a big problem that it's worth spending hours writing documents and getting consensus about a standard API?

As far as I know, there exists 0 dApps on top of PolkadotApi or Subxt right now, so even if it was worth the effort, does this problem really need to be solved now? The further you go into the "standard" side, the less freedom you have to develop PolkadotApi/Subxt. Again, you're just making your life harder.

[xlc]

From a dApp developer point of view, it will be good if I only need to integrate with a single API and automatically supports all the wallets and extensions that confirm to this API.

[tomaka]

automatically supports all the wallets and extensions that confirm to this API.

That's not what this is about, though.

If you want a standard interface between wallets/extensions and dApps, then what needs to be standardized is the list of messages described here: https://github.com/paritytech/substrate-connect/blob/main/packages/connect-extension-protocol/src/index.ts

And not any specific JavaScript API.

[jsdw]

Just to add my current understanding/thoughts of this:

Currently, browser extensions model themselves off an interface created in PolkadotJS which has some issues (eg it doesn't support arbitrary signed extensions and generally the format is a pain to adhere to). Discovering browser extensions is also a bit of a pain; you wait until some value is added to a global object, which you might miss or whatever.

So, I can see value in:

• Defining a better interface to communicate with browser extensions (I think with the expectation that browser extensions can contain either account management bits, and/or a light client from which to obtain metadata and such; possibly like the new JSON-RPC interface one might impl one or both of these parts).
• Defining an approach for discovering browser extensions that implement this interface (eg @josepot talked about an ACK style thing whereby extensions announce themselves via events, and a user can emit an event to ask which extensions exist).

I can see that such an interface could be defined in the form of a TypeScript interface that would be implemented by some objects that browser extensions make available (or whatever else), but it could also be defined in terms of a JSON-RPC style interface (or anything else, really) if we wanted to be a bit more language agnostic (this would then define the structure of messages passed and received, rather than the interface of some object you can get hold of).

I don't see this as an attempt to give Subxt and PAPI a similar API or anything like that. This is an interface that one might call into if one wants to write a Rust-compiled-to-WASM UI making use of Subxt, and then wants to use browser extensions to help sign and/or submit transactions as part of this. I think it'd be easier using this proposed interface than using the current PolkadotJS inspired one in this case. That is a bit of a niche use case I know.

[tomaka]

What the opening post mentions is:

library-agnostic API for dApps to interact seamlessly with the Polkadot network

a standardized protocol for a node and keyring interactions

It doesn't mention more, such as at what level that API would be. I interpret this as a way for dApps to talk to PolkadotApi.

If there is some confusion or disagreement about what should be in this API (let me know if it's just me), then it further reinforces my point that it's too premature to solve this problem now.

---

But this whole discussion is off-topic to the original question, which is whether this fits in the scope of RFCs. And my opinion, whatever this API is, is that no, it doesn't fit in the scope of RFCs, for the reason that this API does in no way need to be unique.

[jsdw]

Yup; let's discuss this on the Polkadot forum and go from there :)

[josepot]

I overall agree with @tomaka on the fact that it's probably too premature to solve this problem now. At this point it probably makes more sense to work on making this a "de-facto" standard, and if we actually manage to make that happen, then perhaps we can think about having a more formal specification around it.

There are a number of things that have been said in here that I would like to clarify, though. However, it's probably best to do that in the polkadot forum 🙂. Anyways, I'm happy that I asked before creating the formal RFC 😅.