jenkins-infra / crawler

tools crawler
26 stars 65 forks source link

fix(codeql.groovy): don't try to convert bundle name as integer #121

Closed lemeurherve closed 2 years ago

lemeurherve commented 2 years ago

The v2.6.0-beta.1 release (published on 2021-07-27, 3 days after the implementation of this particular crawler 😅) has a different download URL pattern than the other, and throw an error on the cast to integer filtering out older versions than 2020-08-26.

While the GitHub releases page returns older releases, the GitHub API used in the script returns only the releases created on the 2021-05-03 or after. I don't think fhe filter isn't needed any more, I can add it back while taking care of the v2.6.0-beta.1 exception if needed.

Fixes https://github.com/jenkins-infra/crawler/issues/122 Fix one of the failing scripts noticed in https://github.com/jenkins-infra/helpdesk/issues/2950

lemeurherve commented 2 years ago

I've checked locally, adding details..

lemeurherve commented 2 years ago

In the current JSON stuck since July 2021 there are more older versions (2.2.5 to 2.5.8) than what's currently retrieved by the GitHub API (2.5.4 to 2.9.4):

downloadService.post('io.jenkins.plugins.codeql.CodeQLInstaller',{"list": [
    {
    "id": "2.5.8",
    "name": "2.5.8",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20210726/"
  },
    {
    "id": "2.5.7",
    "name": "2.5.7",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20210702/"
  },
    {
    "id": "2.5.6",
    "name": "2.5.6",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20210622/"
  },
    {
    "id": "2.5.5",
    "name": "2.5.5",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20210517/"
  },
    {
    "id": "2.5.4",
    "name": "2.5.4",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20210503/"
  },
    {
    "id": "2.5.3",
    "name": "2.5.3",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20210430/"
  },
    {
    "id": "2.5.2",
    "name": "2.5.2",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20210421/"
  },
    {
    "id": "2.5.0",
    "name": "2.5.0",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20210326/"
  },
    {
    "id": "2.4.6",
    "name": "2.4.6",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20210319/"
  },
    {
    "id": "2.4.5",
    "name": "2.4.5",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20210308/"
  },
    {
    "id": "2.4.4",
    "name": "2.4.4",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20210304/"
  },
    {
    "id": "2.4.2",
    "name": "2.4.2",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20210127/"
  },
    {
    "id": "2.4.0",
    "name": "2.4.0",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20201127/"
  },
    {
    "id": "2.3.3",
    "name": "2.3.3",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20201106/"
  },
    {
    "id": "2.3.2",
    "name": "2.3.2",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20201028/"
  },
    {
    "id": "2.3.0",
    "name": "2.3.0",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20201008/"
  },
    {
    "id": "2.2.5",
    "name": "2.2.5",
    "url": "https://github.com/github/codeql-action/releases/download/codeql-bundle-20200826/"
  }
]})

Should I add an hack similar to the jdk one to merge the result from GitHub API with an hardcoded JSON containing v2.2.5 to v2.5.3 data so these previous versions keep being available?

timja commented 2 years ago

🤷 the plugin has 50 installs. Simpler to just use the API I think, but we can add the other option too if needed