search

polkadot-js / api

Promise and RxJS APIs around Polkadot and Substrate based chains via RPC calls. It is dynamically generated based on what the Substrate runtime provides in terms of metadata.
Apache License 2.0
1.07k stars 354 forks source link

Max length for Bytes escape hatch #5990

Open MOZGIII opened 2 months ago

MOZGIII commented 2 months ago

2670 added a hard-coded limit to the max size of the Bytes struct; our runtime exceeds that size.

It is really a bad idea to have hard-coded limits like this, with no escape hatch; a better way to handle that would be an optional constructor argument for the max length with a default value; that way the said default value could be smaller, but adjusted when the context dictates it has to be bigger, or lifted entirely.

Like in our case - we are building some tooling to help us with runtime upgrade testing automation, and we really don't need any sort of fuss from the client-side like this at all - so we'd disable the max length and see where it gets us.

TarikGul commented 2 months ago

If i remember correctly those limits were placed because of some limitation in the extension.

That being said, I agree with the "escape hatch", but there are limitations to some implementations, and the tech debt they bring on when there are so many layers to something.

A small repo would help me understand how you are using the api, which would dictate what the potential solution would look like (short term of long term).

TarikGul commented 2 months ago

In addition to that if this is giving your team to much hassle there is also the alternative which is polkadot-api - which might not have any issue with this.

MOZGIII commented 2 months ago

Here's the gist of it: https://gist.github.com/MOZGIII/1fe5618aaea930bfa00fff06e24fa153

We have an internal repo that has a bunch of cli scripts to support the chain development and operations.

Note now on line 121 (https://gist.github.com/MOZGIII/1fe5618aaea930bfa00fff06e24fa153#file-sample-ts-L121) we have to do some stupid conversions instead of directly passing to bytes? This issue is the reason.

In addition to that if this is giving your team to much hassle there is also the alternative which is polkadot-api - which might not have any issue with this.

We want to use the tools we officially recommend for use with our chain as much as possible - to find the issues like these for instance. Currently we use polkadot.js, but we did consider switching - to https://github.com/dedotdev/dedot mainly, as we don't support light clients yet. Maybe we will - we have quite a bunch of apps to rewrite though, as we won't be using a zoo of tools. So, for now it is easier to keep improving polkadot.js than to switch.