check_service broken in 1.1.1 release

[dfateyev]

After upgrade from 1.0.0 to 1.1.1 version, the "check_service" daemon throws errors in logs:

Feb 23 06:51:42 xxx python3.6[14374]: [anitya.config INFO] Loading Anitya configuration from /etc/anitya/anitya.toml
Feb 23 07:46:46 xxx python3.6[14374]: [anitya ERROR] 'NoneType' object is not subscriptable
Feb 23 07:46:46 xxx python3.6[14374]: Traceback (most recent call last):
Feb 23 07:46:46 xxx python3.6[14374]: File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/anitya/check_service.py", line 223, in run
Feb 23 07:46:46 xxx python3.6[14374]: future.result()
Feb 23 07:46:46 xxx python3.6[14374]: File "/opt/rh/rh-python36/root/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
Feb 23 07:46:46 xxx python3.6[14374]: return self.__get_result()
Feb 23 07:46:46 xxx python3.6[14374]: File "/opt/rh/rh-python36/root/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
Feb 23 07:46:46 xxx python3.6[14374]: raise self._exception
Feb 23 07:46:46 xxx python3.6[14374]: File "/opt/rh/rh-python36/root/usr/lib64/python3.6/concurrent/futures/thread.py", line 56, in run
Feb 23 07:46:46 xxx python3.6[14374]: result = self.fn(*self.args, **self.kwargs)
Feb 23 07:46:46 xxx python3.6[14374]: File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/anitya/check_service.py", line 106, in update_project
Feb 23 07:46:46 xxx python3.6[14374]: utilities.check_project_release(project, session)
Feb 23 07:46:46 xxx python3.6[14374]: File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/anitya/lib/utilities.py", line 87, in check_project_release
Feb 23 07:46:46 xxx python3.6[14374]: versions_prefix = backend.get_versions(project)
Feb 23 07:46:46 xxx python3.6[14374]: File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/anitya/lib/backends/github.py", line 173, in get_versions
Feb 23 07:46:46 xxx python3.6[14374]: versions = cls._retrieve_versions(owner, repo, project)
Feb 23 07:46:46 xxx python3.6[14374]: File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/anitya/lib/backends/github.py", line 129, in _retrieve_versions
Feb 23 07:46:46 xxx python3.6[14374]: versions = parse_json(json, project)
Feb 23 07:46:46 xxx python3.6[14374]: File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/anitya/lib/backends/github.py", line 250, in parse_json
Feb 23 07:46:46 xxx python3.6[14374]: version["version"] = hook["name"]
Feb 23 07:46:46 xxx python3.6[14374]: TypeError: 'NoneType' object is not subscriptable

[Zlopez]

The new version is already running on https://release-monitoring.org/ and I don't see any issue like this in the logs. Do you know which project is causing this?

[dfateyev]

Well, it's hard to determine it right away (since we have more than a hundred of projects monitored). It looks like the Github backend is causing issues — I will add more logging to "check_service" itself to catch the project name, and let you know.

[Zlopez]

You should be able to see this on INFO level I have it enabled on https://release-monitoring.org/

[dfateyev]

Seems like I have found the culprit (in my installation):

Feb 23 10:53:21 xxx python3.6[23721]: [anitya.lib.backends.github DEBUG] Received 10 releases for Dragonwell 8
Feb 23 10:53:21 xxx python3.6[23721]: [anitya ERROR] 'NoneType' object is not subscriptable
Feb 23 10:53:21 xxx python3.6[23721]: Traceback (most recent call last):
Feb 23 10:53:21 xxx python3.6[23721]: File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/anitya/check_service.py", line 223, in run
Feb 23 10:53:21 xxx python3.6[23721]: future.result()
Feb 23 10:53:21 xxx python3.6[23721]: File "/opt/rh/rh-python36/root/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
Feb 23 10:53:21 xxx python3.6[23721]: return self.__get_result()
Feb 23 10:53:21 xxx python3.6[23721]: File "/opt/rh/rh-python36/root/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
Feb 23 10:53:21 xxx python3.6[23721]: raise self._exception
Feb 23 10:53:21 xxx python3.6[23721]: File "/opt/rh/rh-python36/root/usr/lib64/python3.6/concurrent/futures/thread.py", line 56, in run
Feb 23 10:53:21 xxx python3.6[23721]: result = self.fn(*self.args, **self.kwargs)
Feb 23 10:53:21 xxx python3.6[23721]: File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/anitya/check_service.py", line 106, in update_project
Feb 23 10:53:21 xxx python3.6[23721]: utilities.check_project_release(project, session)
Feb 23 10:53:21 xxx python3.6[23721]: File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/anitya/lib/utilities.py", line 87, in check_project_release
Feb 23 10:53:21 xxx python3.6[23721]: versions_prefix = backend.get_versions(project)
Feb 23 10:53:21 xxx python3.6[23721]: File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/anitya/lib/backends/github.py", line 173, in get_versions
Feb 23 10:53:21 xxx python3.6[23721]: versions = cls._retrieve_versions(owner, repo, project)
Feb 23 10:53:21 xxx python3.6[23721]: File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/anitya/lib/backends/github.py", line 129, in _retrieve_versions
Feb 23 10:53:21 xxx python3.6[23721]: versions = parse_json(json, project)
Feb 23 10:53:21 xxx python3.6[23721]: File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/anitya/lib/backends/github.py", line 250, in parse_json
Feb 23 10:53:21 xxx python3.6[23721]: version["version"] = hook["name"]

[Zlopez]

I tried this in GitHub GraphQL API explorer and it worked for this project. You can try it by yourself by using https://docs.github.com/en/graphql/overview/explorer and following query (this is the exact query sent by Anitya):

{
  repository(owner: "alibaba", name: "dragonwell8") {
    releases(orderBy: {field: CREATED_AT, direction: ASC}, last: 50) {
      totalCount
      edges {
        node {
          name
          tag {
            name
            target {
              commitUrl
            }
          }
        }
      }
    }
  }
}

The issue is actually in this release

"node": {
              "name": "Alibaba Dragonwell 8.0-preview",
              "tag": null
            }

This release is not linked to any tag, which is why Anitya is crashing on it. I can fix this in Anitya and skip the releases missing a tag. But this looks like something strange in GitHub API, because there is no release like this in the repository, so I'm not sure where this come from.

But this is definitely a bug.

[Zlopez]

If you want a quick fix for now, just disable the check Check releases instead of tags.

[dfateyev]

Thanks for checking, it definitely looks like something is stuck on Github side. I also think that we should skip such releases, just notifying user about such discrepancies with "WARN" or "ERROR" in logs.

[Zlopez]

I will add the message when I will work on this.