[Milestone] Finish APIv2 design

[kuzdogan]

We are planning a new API for the next quarter: #1367 The goal of this milestone is to think about the how the API should look like and finalize its design.

Background

Some problems with the current API:

• VIP (Very important problem): Users have to wait for a response after submitting a verification. Sometimes verifications can take minutes and this means the request are left hanging for a long time. This is not good for both the server load and the user experience.
• VIP: The "perfect" vs "partial" naming is confusing! Users get confused when they see a "partial" match and assume something went wrong. As if this is not enough, the codebase is also not consistent in the naming of the better match: sometimes it's called "perfect" sometimes "full". Example confusion [1] [2]
• Current API returns an array of results. This is something we inherited because in the old days it was possible to submit multiple chain-address pairs. IMO the API should be simple and only accept a single chain-address pair.
  Request example

{
  "result": [
    {
      "address": "0x123f681646d4a755815f9cb19e1acc8565a0c2ac",
      "chainId": "1",
      "status": "perfect",
      "libraryMap": {
        "lib1": "0x3f681646d4a755815f9cb19e1acc8565a0c2ac",
        "lib2": "0x4f681646d4a755815f9cb19e1acc8565a0c2ac"
      }
    }
  ]
}

• The API accepts both application/json and multipart/form-data which causes confusion and unnecessary overhead IMO.
• Everything’s based on the legacy filesystem of the server e.g. endpoints like:
  • /files/any/{chain}/{address} : the response is just the content of all files under this directory
  • /repository/contracts/full_match/1/0xadb2...cd1f: Based on a static mount of the filesystem directory
  • /files/tree : based on fs, lists all directories
• Everything is based on the Solidity Metadata and all other endpoints (Etherscan, Standard-Input…) are just workarounds to generate the metadata and verify with that. IMO we should move away from basing the verification on the metadata and move to standard JSON. I’d argue mainly for:
  1. There are multiple bugs in the Solidity compiler
     1. https://github.com/ethereum/solidity/issues/14829
     2. https://github.com/ethereum/solidity/issues/14494
     3. https://github.com/ethereum/solidity/issues/14250 where it wasn’t possible to reproduce the same bytecode because of added extra files. These extra files (that are not needed in the compilation of that contract) don’t end up in the metadata.json (rightfully so) but they still affect the bytecode.
  2. To support other languages, we’d need to support standard-json
  3. Metadata file has it’s own problems, mainly around reproducability, and not really liked by the all of the community.
• The session API is not used much and is additional code to maintain. Our main funnel is the API and tooling integrations. The UI is a nice to have but doesn't have to have a session.
• The API does not have understandable errors. We return mostly appropriate HTTP error codes alongside a message but a message is not a good explainer for machine reading purposes. For this, we should return custom error codes inside the code field of the response body to indicate specific errors. See Stripe's error codes for an example. There should be a separate error codes page in the docs to explain different codes.

Open Questions

• How to respond to the user when a contract that is already "partial"ly verified, and the user sends another verification request that yields a "partial" again. We can't respond by just doing a contract lookup and responding yees. We always need to compile and try to re-verify as there's a possibility the new request yields "perfect". Here there are two approaches:
  • I'd expect a 200 Success because my contract is verified
  • I'd expect a non-200 (e.g. 409 Conflict) because the contract in my new request won't be saved.

### Tasks
- [ ] https://github.com/ethereum/sourcify/issues/1545