feat: add option to output metadata to a file

[trusch]

This adds a --output flag to the metadata subcommand. If specified, the prefixed scale-encoded metadata is written to the file as expected by for example subxt or the polkadotjs/api augmentation scripts. If the global --json flag is set, the json of the prefixed metadata is written to the file.

The normal json output is not changed to not break backward compability.

We need this because we want to expose the metadata of our chain directly from CI and don't want to spin up a local parachain setup just to query the metadata from a live node.

Ps: I really like this tool, getting the metadata directly from the wasm is super nice!

edit: Fixes https://github.com/chevdor/subwasm/issues/58

[trusch]

Currently (in this PR) subwasm meta --output <output-file> --json <wasm-blob> would write the json encoded version of the metadata into

and would not store the metadata as hex encoded scale within a json object..

But actually I like the idea of combining --json and --scale to produce something like {"result": "0xdeadbeaf..."}, thats exactly the format that polkadotjs expects.

Ok, let summarize:

• subwasm meta <wasm>
  • outputs json formatted RuntimeMetadata to stdout
• subwasm meta --json <wasm>
  • same as above
• subwasm meta --scale <wasm>
  • outputs hex(scale(prefixed_metadata)), i.e. 0xdeadbeaf...
• subwasm meta --scale --json <wasm>
  • outputs a json object with result key set to hex(scale(prefixed_metadata)), i.e. {"result": "0xdeadbeaf..."}
• subwasm meta --output <file> <wasm>
  • outputs json formatted RuntimeMetadata to
• subwasm meta --output <file> --json <wasm>
  • same as above
• subwasm meta --output <file> --scale <wasm>
  • outputs scale(prefixed_metadata) to
  • this is the format expected by subxt
• subwasm meta --output <file> --scale --json <wasm>
  • outputs a json object with result key set to hex(scale(prefixed_metadata)) to , i.e. {"result": "0xdeadbeaf..."}
  • this is the format expected by polkadotjs

If we agree on that I'll restructure the main to reflect that :)

[chevdor]

• subwasm meta <wasm>

  • outputs json formatted RuntimeMetadata to stdout

No this is not the current behavior, currently it outputs a human friendly list of pallets and the user can dig in using an extra -m (for module)

• subwasm meta --json <wasm>

  • same as above

Not same as above, this one shows the raw json version

• subwasm meta --scale <wasm>

  • outputs hex(scale(prefixed_metadata)), i.e. 0xdeadbeaf...

yep, that would be great, I guess you mean to stdout here

• subwasm meta --scale --json <wasm>

  • outputs a json object with result key set to hex(scale(prefixed_metadata)), i.e. {"result": "0xdeadbeaf..."}

fine by me

• subwasm meta --output <file> <wasm>

  • outputs json formatted RuntimeMetadata to

We may drop --output totally I think, and we let users do subwasm meta ... > output.ext.

• subwasm meta --output <file> --json <wasm>

  • same as above

any "same than above" is likley not required

• subwasm meta --output <file> --scale <wasm>

  • outputs scale(prefixed_metadata) to

see above

• this is the format expected by subxt
  • subwasm meta --output <file> --scale --json <wasm>

can we not drop --output and simply use:

subwasm meta --scale --json <wasm> > <file>

• outputs a json object with result key set to hex(scale(prefixed_metadata)) to , i.e. {"result": "0xdeadbeaf..."}
• this is the format expected by polkadotjs

If we have several "format" for --scale we may be better off using: -- scale <format> with romat one of: subxt | polkadotjs (or whatever names...)

[trusch]

I just discovered that there are even emojis in the human readable output when specifying a module, lfg! Ok, new UX proposal:

We have a flag --format <human | json | scale | json+scale | hex+scale > that defaults to human and we always print to stdout (eventually refusing to write raw scale to a terminal). Thats the least confusing in my opinion.

Does that sound better?

[chevdor]

Yes that sounds very good actually 👍

I indeed would like to keep human as default. Automation can do the effort of adding flags but humans are usually lazy :)

I am not against some --output file, it was so far just not required but I see it could benefit some use case and prevent a few subwasm.... | tee <file>. If we introduce it, it should however just do what users expect: send the output (whaetver it is) to the provided file, and not make any assumption on the format.

In theory, you may want to even do stuff like --format human --output /tmp/foo.bar

Having such a new output flag, we could make it mandatory for raw scale and set a default filename such as metadata_scale_raw.bin to keep it simple for instance.

Cherry on top, for the default file, we may want to keep track of what is the appropriate extension for a given format. For instance:

• human: .txt
• json: .json
• scale: .scale
• scale/json: .jscale ?
• scale/hex: .hscale ?

Those will be user overridable anyway to we help making it easy without getting in the way.

[trusch]

@chevdor that took a little more effort than anticipated, but besides a few minor warnings (error handling in those macros is hard) it should now do exactly what we want.

Only thing I don't know what to do is with the --json flag. Should we respect it for backward compability and translate it to "--format json"?

[chevdor]

@chevdor that took a little more effort than anticipated, but besides a few minor warnings (error handling in those macros is hard) it should now do exactly what we want.

Yes, I do see the extra work, thanks a lot for that !

Only thing I don't know what to do is with the --json flag. Should we respect it for backward compability and translate it to "--format json"?

Yes I want to be extra cautious with that since parity and probably other users may be counting on that. We could either:

• deprecate --json, ie we still support it but show a fat warning mentioning to switch to --format json but keep it for now
• simply keep --json as alias for --format json but... meh... :)

I think the first option is cleaner.

[trusch]

@chevdor Broken Pipe errors are now ignored again + there is now a warning about the deprecation of --json when you use it in the context of retrieving metadata. Error handling is also much better now.

Hope you like it now :)

[chevdor]

Nice work on that PR @trusch, thank you!

[dzmitry-lahoda]

tried to use output of subwasm as input to subxt. not worked.

[chevdor]

Can you please open a new issue and provide more details ? I just found out that this PR introduces a regression that I am fixing now. If this is the issue you are running into, I may have a workaround for you for now.

[dzmitry-lahoda]

https://github.com/chevdor/subwasm/issues/69