zonemaster / zonemaster-backend

The Zonemaster Backend - part of the Zonemaster project
Other
13 stars 23 forks source link

RPCAPI method "get_test_history" includes tests not yet run #888

Closed matsduf closed 2 years ago

matsduf commented 2 years ago

If tests are queued to be run, but not yet started, they will still be included in get_test_history. The documentation says

Returns a list of completed tests for a domain.

Furthermore, the not started tests have overall_result "ok":

When the test was created but before it was run:

    {
      "undelegated": false,
      "creation_time": "2021-10-08 14:10:16",
      "overall_result": "ok",
      "id": "19e491e93e2994d5"
    },

After it was run:

    {
      "creation_time": "2021-10-08 14:10:16",
      "undelegated": false,
      "overall_result": "critical",
      "id": "19e491e93e2994d5"
    },

How to replicate this:

  1. Create API user and API key
  2. Run the follwing command
    zmb get_test_history --domain xi | jq .

    which will probably give a JSON without old tests

    {
    "id": 1,
    "jsonrpc": "2.0",
    "result": []
    }
  3. Run a batch command where domain xi is repeated after some other domains are entered. E.g.:
    zmb --verbose add_batch_job  --username USER  --api-key APIKEY --domain arpa --domain iis.se --domain afnic.fr --domain errors.com --domain icann.com --domain xi --domain xi --domain xi --domain xi --domain xi --domain xi --domain xi --domain xi --domain xi --domain xi --domain xi
  4. Immediately run the following command and check the overall status. All tests are listed with status "ok".
    zmb get_test_history --domain xi | jq .
  5. When the batch has been completed (zmb get_batch_job_result --batch-id BATCH-ID) run the previous get_test_history command again. Now the overall_status is "critical" (since that TLD does not exist).
matsduf commented 2 years ago

Resolved by #942.