Error publishing when solidity compiler settings contain a library remapping

[danceratopz]

Environment information

• ape and plugin versions:

$ ape --version
0.5.4

$ ape plugins list
Installed Plugins:
  solidity     0.5.1
  fantom       0.5.1
  arbitrum     0.5.1
  infura       0.5.2
  polygon      0.5.1a1
  etherscan    0.5.2
  optimism     0.5.1a1

• Python Version: 3.10.6
• OS: Linux Mint 21 (Ubuntu 22.04, jammy)

What went wrong?

If ape-config.xml contains a library remapping such as:

solidity:
  import_remapping:
    - "@openzeppelin/contracts=OpenZeppelin/4.7.2/"

and a previously deployed contract is published with the command:

address = "0x..."
etherscan = networks.provider.network.explorer
etherscan.publish_contract(address)

it fails with the error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In [8], line 1
----> 1 etherscan.publish_contract("0xD71f05258AEE8d6520186D736A64B3525A255395")

File ~/.pyenv/versions/310-circle-nft/lib/python3.10/site-packages/ape_etherscan/explorer.py:71, in Etherscan.publish_contract(self, address)
     69 def publish_contract(self, address: AddressType):
     70     verifier = SourceVerifier(address, self._client_factory)
---> 71     return verifier.attempt_verification()

File ~/.pyenv/versions/310-circle-nft/lib/python3.10/site-packages/ape_etherscan/verify.py:253, in SourceVerifier.attempt_verification(self)
    251 license_code = self.license_code
    252 license_code_value = license_code.value if license_code else None
--> 253 guid = self._contract_client.verify_source_code(
    254     source_code,
    255     compiler_used.version,
    256     contract_name=f"{self._contract_type.source_id}:{self._contract_type.name}",
    257     optimization_used=optimized,
    258     optimization_runs=runs,
    259     constructor_arguments=self.constructor_arguments,
    260     evm_version=evm_version,
    261     license_type=license_code_value,
    262 )
    263 self._wait_for_verification(guid)

File ~/.pyenv/versions/310-circle-nft/lib/python3.10/site-packages/ape_etherscan/client.py:210, in ContractClient.verify_source_code(self, standard_json_output, compiler_version, contract_name, optimization_used, optimization_runs, constructor_arguments, evm_version, license_type, libraries)
    195 if not compiler_version.startswith("v"):
    196     compiler_version = f"v{compiler_version}"
    198 json_dict = {
    199     **self.base_params,
    200     "action": "verifysourcecode",
    201     "codeformat": "solidity-standard-json-input",
    202     "compilerversion": compiler_version,
    203     "constructorArguements": constructor_arguments,
    204     "contractaddress": self._address,
    205     "contractname": contract_name,
    206     "evmversion": evm_version,
    207     "licenseType": license_type,
    208     "optimizationUsed": int(optimization_used),
    209     "runs": optimization_runs,
--> 210     "sourceCode": io.StringIO(json.dumps(standard_json_output)),
    211 }
    213 iterator = 1
    214 for lib_address, lib_name in libraries.items():

File /usr/lib/python3.10/json/__init__.py:231, in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    226 # cached encoder
    227 if (not skipkeys and ensure_ascii and
    228     check_circular and allow_nan and
    229     cls is None and indent is None and separators is None and
    230     default is None and not sort_keys and not kw):
--> 231     return _default_encoder.encode(obj)
    232 if cls is None:
    233     cls = JSONEncoder

File /usr/lib/python3.10/json/encoder.py:199, in JSONEncoder.encode(self, o)
    195         return encode_basestring(o)
    196 # This doesn't pass the iterator directly to ''.join() because the
    197 # exceptions aren't as detailed.  The list call should be roughly
    198 # equivalent to the PySequence_Fast that ''.join() would do.
--> 199 chunks = self.iterencode(o, _one_shot=True)
    200 if not isinstance(chunks, (list, tuple)):
    201     chunks = list(chunks)

File /usr/lib/python3.10/json/encoder.py:257, in JSONEncoder.iterencode(self, o, _one_shot)
    252 else:
    253     _iterencode = _make_iterencode(
    254         markers, self.default, _encoder, self.indent, floatstr,
    255         self.key_separator, self.item_separator, self.sort_keys,
    256         self.skipkeys, _one_shot)
--> 257 return _iterencode(o, 0)

File /usr/lib/python3.10/json/encoder.py:179, in JSONEncoder.default(self, o)
    160 def default(self, o):
    161     """Implement this method in a subclass such that it returns
    162     a serializable object for ``o``, or calls the base implementation
    163     (to raise a ``TypeError``).
   (...)
    177 
    178     """
--> 179     raise TypeError(f'Object of type {o.__class__.__name__} '
    180                     f'is not JSON serializable')

TypeError: Object of type set is not JSON serializable
> /usr/lib/python3.10/json/encoder.py(179)default()
    177 
    178         """
--> 179         raise TypeError(f'Object of type {o.__class__.__name__} '
    180                         f'is not JSON serializable')
    181

Upon inspection in the debugger it seems to be caused by an import_remapping config in ape-config.xml (as included above):

ipdb> o
{'@openzeppelin/contracts=.cache/OpenZeppelin/v4.7.2'}

which is of Python type set.

We can see that the standard_json_output provided to json.dumps (in ape_etherscan/client.py:210) contains a settings dict with a top-level "remappings" entry whose value is a set:

{
    "language": "Solidity",
    "sources": {
        "MyContract.sol": {
            "content": ' ...'
        }
    },
    "settings": {
        "optimizer": {
            "enabled": True,
            "runs": 200
        },
        "outputSelection": {
            "Strings.sol": {
                "Strings": [
                    "abi",
                    "bin",
                    "bin-runtime",
                    "devdoc",
                    "userdoc"
                ]
            }
           <-- snip -->
        },
        "remappings": {
            "@openzeppelin/contracts=.cache/OpenZeppelin/v4.7.2"
        },
    },
}

The settings are obtained from the compiler's settings in attempt_verification() from verify.py.

How can it be fixed?

The remappings entry in the settings should be a list, for example

{
  "remappings": [
    "ds-test/=lib/ds-test/src/",
    "forge-std/=lib/forge-std/src/"
  ],
  <-- snip -->

ape-etherscan should probably do some type checking or casting before submitting the published settings. We could check if there's any advantage to holding the "remappings" in a set in https://github.com/ApeWorX/ape-solidity, if not, it could potentially also be changed there.

[antazoey]

The settings in get_compiler_settings should return compliant JSON for publishing; that was the intent anyway. Having validation in ape-etherscan is good to though, but the main bug is ape-solidity so I am going to fix it there.

thank you for reporting and sorry it took so long to get to this!

[antazoey]

ape-solidity PR: https://github.com/ApeWorX/ape-solidity/pull/79/files