zeta-chain / node

ZetaChain’s blockchain node and an observer validator client
https://zetachain.com
MIT License
164 stars 107 forks source link

Artillery based stress testing is inaccurate due to misconfiguration #2598

Closed CryptoFewka closed 1 month ago

CryptoFewka commented 3 months ago

Describe the Bug The artillery config uses at least one call which is improperly specified: https://github.com/zeta-chain/node/blob/6e9dc53ebc590b82cf5ef48707d847464fd87a21/.github/actions/performance-tests/art.yaml#L482-L486

This call results in HTTP status 200, but a body of:

{
  "jsonrpc": "2.0",
  "id": 1,
  "error": {
    "code": -32602,
    "message": "invalid argument 0: invalid address: hex string of odd length"
  }
}

To Reproduce

Expected Behavior The intent of the artillery stress test is to judge performance of individual methods. This intent is not met with the current configuration, and it instead measures the ability to recognize invalid input and respond with an error message quickly.

Environment (please complete the following information):

CryptoFewka commented 3 months ago

Perhaps we should add and use the expect plugin to ensure that the response doesn't contain an error property. I do not believe that the assert plugin exists, despite an attempt to define its config to try and catch this exact state.

Ref:

Example Response:

{
  "jsonrpc": "2.0",
  "id": 1,
  "error": {
    "code": -32602,
    "message": "invalid argument 0: invalid address: hex string of odd length"
  }
}

Example config (last line contains change) that I would expect to fail on that response:

  - name: eth_getLogs
    flow:
      - post:
          url: "/"
          json:
            id: 1
            jsonrpc: "2.0"
            method: "eth_getLogs"
            params:
              - fromBlock: "0x1"
                toBlock: "latest"
                address: "0x0"
                topics: ["0x0"]
          capture:
            - json: "$"
              as: "response"
          ensure:
            - statusCode: 200
            - statusCode: 201
          expect:
            - notHasProperty: 'data[0].error'