search

w3c / spec-prod

GitHub Action to build ReSpec/Bikeshed specs, validate output and publish to GitHub pages or W3C
https://w3c.github.io/spec-prod/
MIT License
34 stars 21 forks source link

Action fails to report some Echidna failures #91

Open tidoust opened 3 years ago

tidoust commented 3 years ago

Publication through Echidna for the device-memory spec failed in the following workflow and yet the worfklow reported success, which surprises me: https://github.com/w3c/device-memory/runs/3002900039?check_suite_focus=true

(FWIW, the problem is that the specStatus parameter was not passed correctly. The problem is not the error itself, it's the fact that the error only appeared in the logs)

Deploy to W3C
  📣 If it fails, check https://lists.w3.org/Archives/Public/public-tr-notifications/
  $ mv index.html Overview.html
  $ tar cvf /tmp/echidna.tar *
    Overview.html
  $ mv Overview.html index.html
  $ curl 'https://labs.w3.org/echidna/api/request' -F "tar=@/tmp/echidna.tar" -F "token=***"
     -F "annotation=triggered by auto-publish spec-prod action on w3c/device-memory"
     -F "decision=https://lists.w3.org/Archives/Public/public-web-perf/2021Apr/0005.html"
  Getting publish status...
    ⏱️ Wait 6s for job to finish...
    📡 Request: https://labs.w3.org/echidna/api/status?id=e5dae51f-0cb7-4a06-83e2-6a32087ee9a0
    ⏱️ Wait 3s for job to finish...
    📡 Request: https://labs.w3.org/echidna/api/status?id=e5dae51f-0cb7-4a06-83e2-6a32087ee9a0
    ⏱️ Wait 2s for job to finish...
    📡 Request: https://labs.w3.org/echidna/api/status?id=e5dae51f-0cb7-4a06-83e2-6a32087ee9a0
    ⏱️ Wait 8s for job to finish...
    📡 Request: https://labs.w3.org/echidna/api/status?id=e5dae51f-0cb7-4a06-83e2-6a32087ee9a0
    ⏱️ Wait 2s for job to finish...
    📡 Request: https://labs.w3.org/echidna/api/status?id=e5dae51f-0cb7-4a06-83e2-6a32087ee9a0
    ⏱️ Wait 5s for job to finish...
    📡 Request: https://labs.w3.org/echidna/api/status?id=e5dae51f-0cb7-4a06-83e2-6a32087ee9a0
    ⏱️ Wait 10s for job to finish...
    📡 Request: https://labs.w3.org/echidna/api/status?id=e5dae51f-0cb7-4a06-83e2-6a32087ee9a0
    ⏱️ Wait 6s for job to finish...
    📡 Request: https://labs.w3.org/echidna/api/status?id=e5dae51f-0cb7-4a06-83e2-6a32087ee9a0
  {
    id: 'e5dae51f-0cb7-4a06-83e2-6a32087ee9a0',
    status: 'error',
    details: 'https://labs.w3.org/echidna/api/status?id=e5dae51f-0cb7-4a06-83e2-6a32087ee9a0',
    deploymentURL: undefined
  }
marcoscaceres commented 3 years ago

@sidvishnoi pointed me to the code, but I've misplaced the link... apparently, we need to check for both "failure" and "error".