Webhook test delivery raises exception

[bilderbuchi]

When using the "Test Delivery" button in the Gitea webhook config screen to test if the hooks come through correctly, this raises an exception in Buildbot.

2022-01-21 09:53:58+0100 [_GenericHTTPChannelProtocol,47,172.20.16.89] Received event 'push' from gitea
2022-01-21 09:53:58+0100 [_GenericHTTPChannelProtocol,47,172.20.16.89] adding changes from web hook
        Traceback (most recent call last):
          File "/home/myuser/.conda/envs/buildbot-test/lib/python3.9/site-packages/buildbot/www/change_hook.py", line 168, in getChanges
            changes, src = yield handler.getChanges(request)
          File "/home/myuser/.conda/envs/buildbot-test/lib/python3.9/site-packages/buildbot_gitea/webhook.py", line 180, in getChanges
            return self.handler.getChanges(request)
          File "/home/myuser/.conda/envs/buildbot-test/lib/python3.9/site-packages/twisted/internet/defer.py", line 1909, in unwindGenerator
            return _cancellableInlineCallbacks(gen)  # type: ignore[unreachable]
          File "/home/myuser/.conda/envs/buildbot-test/lib/python3.9/site-packages/twisted/internet/defer.py", line 1816, in _cancellableInlineCallbacks
            _inlineCallbacks(None, gen, status)
        --- <exception caught here> ---
          File "/home/myuser/.conda/envs/buildbot-test/lib/python3.9/site-packages/buildbot/www/change_hook.py", line 106, in getAndSubmitChanges
            changes, src = yield self.getChanges(request)
          File "/home/myuser/.conda/envs/buildbot-test/lib/python3.9/site-packages/buildbot/www/change_hook.py", line 168, in getChanges
            changes, src = yield handler.getChanges(request)
          File "/home/myuser/.conda/envs/buildbot-test/lib/python3.9/site-packages/twisted/internet/defer.py", line 1661, in _inlineCallbacks
            result = current_context.run(gen.send, result)
          File "/home/myuser/.conda/envs/buildbot-test/lib/python3.9/site-packages/buildbot_gitea/webhook.py", line 162, in getChanges
            changes = handler_function(payload, event_type, codebase)
          File "/home/myuser/.conda/envs/buildbot-test/lib/python3.9/site-packages/buildbot_gitea/webhook.py", line 44, in process_push
            files.extend(commit.get(kind, []))
        builtins.TypeError: 'NoneType' object is not iterable

I noticed that in the test request, the contained commit has

      "added": null,
      "removed": null,
      "modified": null

In conventional commits, these are either [] or lists-of-strings.

I think I can confirm the behaviour leading to the exception raised here like this:

>>> commit = json.loads('{"added": null}')
>>> print(commit.get("added", []))
None

Becaus the added key exists, the [] default is not used. It looks like the logic here needs to be adapted to turn a None/falsy value into an empty list, maybe like this: files.extend(commit.get("added", []) or []).

Versions

Python version: 3.9.9 Buildbot version: 3.4.0 Twisted version: 21.7.0 buildbot-gitea: 1.7.0

[pampersrocker]

Thank you for the report.

Do you have the (anonymized) webhook content which causes this exception?

[bilderbuchi]

Not anonymized, no, that's why I posted the portion that looks relevant. To you want me to manually anonymize and post the content?

[pampersrocker]

I mean the data gitea sends when you push the test webhook button. As this uses one of your last commits in the repo for sending.

[pampersrocker]

I would use this to create a unit test for the webhook data.

[bilderbuchi]

This should do it:

{
  "ref": "refs/heads/develop",
  "before": "2437bd7c6b0af7b8da570973c02f0cca07ec787d",
  "after": "2437bd7c6b0af7b8da570973c02f0cca07ec787d",
  "compare_url": "",
  "commits": [
    {
      "id": "2437bd7c6b0af7b8da570973c02f0cca07ec787d",
      "message": "snip",
      "url": "snip/commit/2437bd7c6b0af7b8da570973c02f0cca07ec787d",
      "author": {
        "name": "snip",
        "email": "snip",
        "username": ""
      },
      "committer": {
        "name": "snip",
        "email": "snip",
        "username": ""
      },
      "verification": null,
      "timestamp": "0001-01-01T00:00:00Z",
      "added": null,
      "removed": null,
      "modified": null
    }
  ],
  "head_commit": {
    "id": "2437bd7c6b0af7b8da570973c02f0cca07ec787d",
    "message": "snip",
    "url": "snip/commit/2437bd7c6b0af7b8da570973c02f0cca07ec787d",
    "author": {
      "name": "snip",
      "email": "snip",
      "username": ""
    },
    "committer": {
      "name": "snip",
      "email": "snip",
      "username": ""
    },
    "verification": null,
    "timestamp": "0001-01-01T00:00:00Z",
    "added": null,
    "removed": null,
    "modified": null
  },
  "repository": {
    "id": snip,
    "owner": {"id":snip,"login":"snip","full_name":"snip","email":"snip","avatar_url":"snip","language":"","is_admin":false,"last_login":"0001-01-01T00:00:00Z","created":"2019-07-04T02:15:26+02:00","restricted":false,"active":false,"prohibit_login":false,"location":"","website":"","description":"","visibility":"public","followers_count":0,"following_count":0,"starred_repos_count":0,"username":"snip"},
    "name": "snip",
    "full_name": "snip",
    "description": "snip",
    "empty": false,
    "private": true,
    "fork": false,
    "template": false,
    "parent": null,
    "mirror": false,
    "size": 19106,
    "html_url": "snip",
    "ssh_url": "git@snip.git",
    "clone_url": "snip.git",
    "original_url": "",
    "website": "",
    "stars_count": 0,
    "forks_count": 0,
    "watchers_count": 5,
    "open_issues_count": 33,
    "open_pr_counter": 1,
    "release_counter": 0,
    "default_branch": "develop",
    "archived": false,
    "created_at": "2020-08-25T09:34:29+02:00",
    "updated_at": "2022-01-19T15:55:11+01:00",
    "permissions": {
      "admin": false,
      "push": false,
      "pull": false
    },
    "has_issues": true,
    "internal_tracker": {
      "enable_time_tracker": false,
      "allow_only_contributors_to_track_time": true,
      "enable_issue_dependencies": true
    },
    "has_wiki": true,
    "has_pull_requests": true,
    "has_projects": false,
    "ignore_whitespace_conflicts": false,
    "allow_merge_commits": true,
    "allow_rebase": true,
    "allow_rebase_explicit": false,
    "allow_squash_merge": true,
    "default_merge_style": "merge",
    "avatar_url": "",
    "internal": false,
    "mirror_interval": ""
  },
  "pusher": {"id":snip,"login":"snip","full_name":"snip","email":"snip","avatar_url":"snip","language":"","is_admin":false,"last_login":"0001-01-01T00:00:00Z","created":"2021-01-22T02:15:29+01:00","restricted":false,"active":false,"prohibit_login":false,"location":"","website":"","description":"","visibility":"public","followers_count":0,"following_count":0,"starred_repos_count":0,"username":"snip"},
  "sender": {"id":snip,"login":"snip","full_name":"snip","email":"snip","avatar_url":"snip","language":"","is_admin":false,"last_login":"0001-01-01T00:00:00Z","created":"2021-01-22T02:15:29+01:00","restricted":false,"active":false,"prohibit_login":false,"location":"","website":"","description":"","visibility":"public","followers_count":0,"following_count":0,"starred_repos_count":0,"username":"snip"}
}

[pampersrocker]

This has been fixed with your recommendation in v1.7.1, thanks again!