Attempt to verify Contract using Hardhat Fails with 500 Error.

[bugbytesinc]

Description

There is a particular contract that fails with a 500 error, see error log below.

Steps to reproduce

Compile contracts using hardhat Attempt to verify

Additional context

$ npx hardhat verify 0x331323d5A987F1Ed9D66f07D3ba22702887726FA --verbose
  hardhat:core:vars:varsManager Creating a new instance of VarsManager +0ms
  hardhat:core:vars:varsManager Loading ENV variables if any +2ms
  hardhat:core:config Loading Hardhat config from /home/jasonf/bonzo-finance-contracts/hardhat.config.ts +0ms
  hardhat:core:hre Creating HardhatRuntimeEnvironment +0ms
  hardhat:core:global-dir Looking up Client Id at /home/jasonf/.local/share/hardhat-nodejs/analytics.json +0ms
  hardhat:core:global-dir Client Id found: 1fbe366e-5d28-4fc0-853f-33e9b3991d0a +1ms
  hardhat:core:analytics Sending hit for task +0ms
  hardhat:core:analytics Hit payload: {"client_id":"1fbe366e-5d28-4fc0-853f-33e9b3991d0a","user_id":"1fbe366e-5d28-4fc0-853f-33e9b3991d0a","user_properties":{"projectId":{"value":"hardhat-project"},"userType":{"value":"Developer"},"hardhatVersion":{"value":"Hardhat 2.22.2"},"operatingSystem":{"value":"linux"},"nodeVersion":{"value":"v18.17.0"}},"events":[{"name":"task","params":{"engagement_time_msec":"10000","session_id":"0.841666911637083"}}]} +1ms
  hardhat:core:hre Running task verify +100ms
  hardhat:core:hre Running task verify:get-verification-subtasks +1ms
  hardhat:core:hre Running task verify:sourcify +0ms
  hardhat:core:hre Running task verify:sourcify-resolve-arguments +0ms
  hardhat:core:hre Creating provider for network hedera_testnet +1ms
  hardhat:core:analytics Hit for task sent successfully +5s
  hardhat:hardhat-verify:metadata Read metadata length 53 +0ms
  hardhat:hardhat-verify:metadata Last 51 bytes of metadata: a26469706673582212209a49528ec29c88d8102a0d92595aa78c843c57b5ac5e2fdfcf0163c123cf037464736f6c634300060c +0ms
  hardhat:hardhat-verify:metadata Metadata decoded: {
  hardhat:hardhat-verify:metadata   ipfs: <Buffer 12 20 9a 49 52 8e c2 9c 88 d8 10 2a 0d 92 59 5a a7 8c 84 3c 57 b5 ac 5e 2f df cf 01 63 c1 23 cf 03 74>,
  hardhat:hardhat-verify:metadata   solc: <Buffer 00 06 0c>
  hardhat:hardhat-verify:metadata } +2ms
  hardhat:hardhat-verify:metadata Solc version detected in bytecode: 0.6.12 +0ms
  hardhat:core:hre Running task verify:get-contract-information +16s
  hardhat:core:hre Running task verify:sourcify-attempt-verification +7s
hardhat-verify found one or more errors during the verification process:

Sourcify:
An unexpected error occurred during the verification process.
Please report this issue to the Hardhat team.
Error Details: Failed to send contract verification request.
Endpoint URL: https://server-verify.hashscan.io/
The HTTP server response is not ok. Status code: 500 Response text: {"error":"stdout maxBuffer length exceeded"}

Hedera network

testnet

Version

Latest

Operating system

Linux

[acuarica]

Hi @bugbytesinc thanks for reporting this. Are you able to provide this Hardhat project so I can reproduce the error?

Thanks.

[acuarica]

Hi @bugbytesinc, after further investigation I suspect the error is raised while compiling the contract. In particular, the output of the solc compiler is larger than the maxBuffer size. https://github.com/ethereum/sourcify/blob/e6859d429522043e61643ade1092887e6da3cfc9/services/server/src/server/services/compiler/local/solidityCompiler.ts#L327-L343

Here's a minimal snippet to reproduce this kind of error (essentially what the verification service is doing to invoke solc)

#!/usr/bin/env node

const { exec } = require('child_process');

if (process.argv.length < 3) {
    exec(`${process.argv[1]} 1`, { maxBuffer: 10 },
        (error, _stdout, _stderr) => {
            if (error) console.log(`Error exception while running child process:`, error);
        }
    );
} else {
    console.log('123456789ab');
}

(for this to work the file for the script must be executable)

If you can a share an example where this happens I could see if that actually the problem and if there is a workaround.

[bugbytesinc]

Trying to see if we can get a clean room example, it will take some doing.

[bugbytesinc]

Apologies, for not getting you an example contract yet, the project is a bit complex and factoring out a working example is proving difficult. However, I was able to try another approach and have discovered pretty strong evidence that indeed the configuration of the solc compiler you mention above is the root of the issue we are having.

I downloaded the hedera-sourcify project and started a local server pointed it at testnet. After making the following change:

diff --git a/services/server/src/server/services/compiler/local/solidityCompiler.ts b/services/server/src/server/services/compiler/local/solidityCompiler.ts
index c62c955a..4be6f775 100644
--- a/services/server/src/server/services/compiler/local/solidityCompiler.ts
+++ b/services/server/src/server/services/compiler/local/solidityCompiler.ts
@@ -328,7 +328,7 @@ function asyncExecSolc(
     const child = exec(
       `${solcPath} --standard-json`,
       {
-        maxBuffer: 1000 * 1000 * 20,
+        maxBuffer: 1024 * 1024 * 40,
       },
       (error, stdout, stderr) => {
         if (error) {

our project validated without error.

[bugbytesinc]

So in summary, it looks to me that we have a situation where the Hedera network will accept contracts of a certain size that the default configuration of the solc configuration of the upstream ethereum sourcify server project did not anticipate and can't handle. I don't know what other considerations there might be for enlarging the buffer size in this case, but it looks like possibly a quick win to me.

[acuarica]

Hey @bugbytesinc, thank you for confirming this.

the project is a bit complex and factoring out a working example is proving difficult.

Are you able to tell how big the project is? How many files, contracts, interfaces, libraries are involved in the compilation, roughly. This is to ensure that we are on the right track. The current maxBuffer is set to 20MB which I assume if big enough for compilation output even if the project is a big one.

On top of that, are you able to provide the output of the solc compiler? If you use a dev environment, e.g., Hardhat or Foundry, do you see a large output during compilation?

[acuarica]

it looks to me that we have a situation where the Hedera network will accept contracts of a certain size that the default configuration of the solc configuration of the upstream ethereum sourcify server project did not anticipate and can't handle.

Just to avoid any future confusion, these two thing are unrelated. The error here is about the size of the stdin input of the solc compiler, not the size of the compiled contracts.

[acuarica]

Looks like upstream Sourcify already ran into this issue. It was solved here https://github.com/ethereum/sourcify/commit/585b241299b8276ccd569baf695ecb96257fca78.

[bugbytesinc]

In our project, running this against https://server-verify.hashscan.io yields:

$ npx hardhat verify 0x2f472b8a10C3F0264c479a31C4d0998836783845
hardhat-verify found one or more errors during the verification process:

Sourcify:
An unexpected error occurred during the verification process.
Please report this issue to the Hardhat team.
Error Details: Failed to send contract verification request.
Endpoint URL: https://server-verify.hashscan.io/
The HTTP server response is not ok. Status code: 500 Response text: {"error":"stdout maxBuffer length exceeded"}

[bugbytesinc]

Running against a version modified with the above patch yields:

npx hardhat verify 0x2f472b8a10C3F0264c479a31C4d0998836783845
Successfully verified contract ValidationLogic on Sourcify.
https://repository-verify.hashscan.io/contracts/full_match/296/0x2f472b8a10C3F0264c479a31C4d0998836783845/

(please note, I didn't change the browserURL configuration so the reference to repository-verify.hashscan.io will not be valid)

[bugbytesinc]

The log file is too large to upload, github will not load it.

[bugbytesinc]

hedera-sourcify-solc-output - Abriged.log Here is an abridged version instead.

[bugbytesinc]

How many files, contracts, interfaces, libraries are involved in the compilation, roughly.

Roughly 141 Solidity files to compile

[bugbytesinc]

are you able to provide the output of the solc compiler?

Please see above, it is too large to upload to github, is there another means we can get it to you if you wish to look at it?

[bugbytesinc]

The current maxBuffer is set to 20MB which I assume if big enough for compilation output even if the project is a big one.

What I have seen is that changing that value to 40MB results in successful validation of our contracts.

[bugbytesinc]

Looks like upstream Sourcify already ran into this issue. It was solved here ethereum/sourcify@585b241.

When I pull the latest hedera sourcify, I get code that has even a smaller value than what is cited in a the upstream project cited above:

maxBuffer: 1000 * 1000 * 20,

hedera-sourcify/sourcify/services/server/src/server/services/compiler/local/solidityCompiler.ts Line 331

[bugbytesinc]

Looks like upstream Sourcify already ran into this issue. It was solved here ethereum/sourcify@585b241.

The Hedera Sourcify source code is pegged to an older version:

https://github.com/ethereum/sourcify/commits/9c0dd54913422ddb1a941af5494be01faa065c7b/ (May 14)

This update is from May 29

https://github.com/ethereum/sourcify/commit/585b241299b8276ccd569baf695ecb96257fca78

Perhaps updating the upstream versions utilized by hedera sourcify might solve our issue.

[acuarica]

The Hedera Sourcify source code is pegged to an older version

Indeed, this is the case. We usually wait until upstream Sourcify makes a new release and is stable enough for us to update.

Perhaps updating the upstream versions utilized by hedera sourcify might solve our issue.

That should be the case. Upstream Sourcify released a new version last week. They made several changes, in particular to use a DB as a storage repository instead of a filesystem. If the new version if compatible then we can update on our side to fix this issue. Otherwise, we will take a bit longer to accommodate for the changes in the new version.