[CLI] Drop --combined-json option and direct users to standard json

[axic]

Closes #10276.

[chriseth]

I think it has some features that standard json lacks.

[axic]

Can you list those? We should add it to standard json.

[chriseth]

Maybe I confused this with non-json mode.

[axic]

Frameworks:

• Waffle: standard-json
• Truffle: standard-json
• Embark: standard-json
• dapptools: uses --combined-json (as of https://github.com/dapphub/dapptools/pull/569 it uses --standard-json)
• brownie: uses py-solc-x uses both --standard-json and --combined-json
• Buidler/Hardhat: uses only --standard-json
• saddle: uses --combined-json as of version 0.1.23. (Tracker: https://github.com/compound-finance/saddle/issues/43)

Tools:

• crytic-compile: uses only --combined-json (Tracker: https://github.com/crytic/crytic-compile/issues/140)
• gasol: input/output in --combined-json asm format.
• ethers-rs: standard-json (has dropped --combined-json support recently).
• mythril: uses --standard-json as of v0.22.32.

I'm sure there are more frameworks, just I'm unaware of them.

[axic]

@iamdefinitelyahuman could you explain whether you rely on --combined-json and if yes, why?

@MrChico likewise, could you explain if moving to --standard-json would be debilitating? :)

[cameel]

Buidler/Hardhat: uses only --standard-json

[axic]

@cameel thanks, added to the list above (please feel free to edit my comment if you check other frameworks)

[iamdefinitelyahuman]

Brownie uses --standard-json exclusively . py-solc-x is just wrapping around solc, so it provides access to --combined-json for no reason other than that combined json exists :)

Brownie would not break anywhere if you removed --combined-json, so :+1: from me.

[axic]

@iamdefinitelyahuman thanks for the response! Do you know of any py-solc-x user who makes use of --combined-json?

[MrChico]

I mainly prefer --combined-json for it's easy commandline use – I can get all of the compiler outputs of interest with a relatively simply one liner. As far as I can foresee there would be nothing fundamentally limiting to migrate dapptools to use --standard-json besides the obvious labor involved in architecturing everything to support the new format. Many features that have been introduced into --standard-json have been neglected in --combined-json, so if that pattern is to continue we would probably migrate anyway. My preference would of course be to simply get --combined-json up to feature parity, as requested in https://github.com/ethereum/solidity/issues/10274.

[axic]

The thing is we have three outputs to maintain:

1. standard-json
2. individual options to solc
3. the legacy combined-json (which predates standard-json)

We can barely keep feature parity between individual solc options and standard-json, and since combined-json is so sparsely used (seemingly only by dapphub), it is hard to rationalise keeping it maintained.

See the various open issues regarding CLI issues.

[iamdefinitelyahuman]

@iamdefinitelyahuman thanks for the response! Do you know of any py-solc-x user who makes use of --combined-json?

I took a look through some of the listed dependents, most are using the standard JSON interface. The notable exception is NuCypher. @KPrasch @cygnusv - care to weigh in here?

[vzotova]

@iamdefinitelyahuman @KPrasch and I are working on that, probably will finish moving to standard JSON interface soon

[axic]

@MrChico could you point me to the places in dapptools where --combined-json is used? I am wondering whether the output parsing or the input preparation seems like the bigger hurdle.

The output should be much more nicely formatted (compared to --combined-json), but at the cost of more levels of objects.

[MrChico]

@axic, it is being called in dapp build and parsed in a few different places, such as dapp create or (most frequently) in Solidity.hs of hevm.

[axic]

@MrChico Looking at dapp build, wouldn't that be better off using the CLI options directly, would they output proper files (e.g. #10275)?

[MrChico]

It does use the CLI options directly if the --extract flag is passed. Not everything that is available under --combined-json is available as a direct CLI flag however, for example--srcmap does not exist. Most of the time, it's quite convenient to have all artifacts in one file though.

[MrChico]

Not everything that is available under --combined-json is available as a direct CLI flag however, for example--srcmap does not exist.

Which I get is part of your original point of the maintenance burden. I can investigate migrating to the --standard-json format when I have some more time (in about 2 weeks).

[MrChico]

OK, having started the process of supporting stdjson, this seems doable. I really miss the pretty printed error messages. I know they are present in the json, but without the nice coloring. To me it would seem appropriate to also display them (pretty printed) in stderr

[axic]

I really miss the pretty printed error messages. I know they are present in the json, but without the nice coloring.

That is coming in 0.8.0.

[MrChico]

@axic nice! Will color be added to the json or output to stderr?

[axic]

The new formatting replaces the old in the formattedMessage field. However the colours are disabled, as of now.

What do you mean by stderr? No error/warning should be reported by the compiler apart from things encoded in the JSON response.

[MrChico]

By stderr I mean the linux standard IO stream, where errors are outputted in any other invocation of solc that doesn't involve --standard-json.

No error/warning should be reported by the compiler apart from things encoded in the JSON response.

I find this a bit unfortunate, but I understand.

[axic]

By stderr I mean the linux standard IO stream, where errors are outputted in any other invocation of solc that doesn't involve --standard-json.

On the regular non-standard-json invocations the stderr was coloured for quite a bit already. That does not change.

[MrChico]

As of https://github.com/dapphub/dapptools/pull/569, dapptools now no longer relies on combined-json.

[axic]

Thanks @MrChico!

[cameel]

The saddle framework is using --combined-json.

EDIT: Added to @axic's list above.

[axic]

Thanks @cameel, opened an issue for them: https://github.com/compound-finance/saddle/issues/43

[axic]

@iamdefinitelyahuman is py-solc-x still dependent on --combined-json?

[cameel]

@axic I have just noticed a problem here. I think that solc --import-ast only accepts input produced by --combined-json ast. The format of --ast-compact-json is different. We should probably provide a replacement before we drop --combined-json.

[cameel]

gasol is another tool that expects input in the --combined-json asm format (and also outputs that format).

Looks like evm.legacyAssembly is basically the same format if you look at each individual contract but contracts are laid out a bit differently in JSON and it's a part of a bigger JSON structure so it needs at least a simple transformation to be usable.