GothenburgBitFactory / bugwarrior

Pull github, bitbucket, and trac issues into taskwarrior
http://pypi.python.org/pypi/bugwarrior
GNU General Public License v3.0
732 stars 209 forks source link

trello: KeyError: `memberCreator` #1004

Open ferdinandyb opened 10 months ago

ferdinandyb commented 10 months ago

I got an error (after waiting for a very long time), trying to sync trello with the following config:

[trello]
service = trello
trello.api_key = [apikey]
trello.token = @oracle:eval:pass show bugwarrior/trello/token

Error:

ERROR:bugwarrior.services:Worker for [trello] failed: 'memberCreator'
Traceback (most recent call last):
  File "/home/fbence/.pyenv/versions/3.11.1/lib/python3.11/site-packages/bugwarrior/services/__init__.py", line 504, in _aggregate_issues
    for issue in service.issues():
  File "/home/fbence/.pyenv/versions/3.11.1/lib/python3.11/site-packages/bugwarrior/services/trello.py", line 123, in issues
    issue.update_extra({"annotations": self.annotations(card)})
                                       ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fbence/.pyenv/versions/3.11.1/lib/python3.11/site-packages/bugwarrior/services/trello.py", line 130, in annotations
    annotations = self.build_annotations(
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/fbence/.pyenv/versions/3.11.1/lib/python3.11/site-packages/bugwarrior/services/__init__.py", line 149, in build_annotations
    for author, message in annotations:
  File "/home/fbence/.pyenv/versions/3.11.1/lib/python3.11/site-packages/bugwarrior/services/trello.py", line 131, in <genexpr>
    ((c['memberCreator']['username'], c['data']['text']) for c in comments),
      ~^^^^^^^^^^^^^^^^^
KeyError: 'memberCreator'
INFO:bugwarrior.services:Done with [trello] in 4556.107841s
INFO:bugwarrior.services:Terminating workers
ERROR:bugwarrior.command:Aborted (critical error in target 'trello')
Traceback (most recent call last):
  File "/home/fbence/.pyenv/versions/3.11.1/lib/python3.11/site-packages/bugwarrior/command.py", line 73, in pull
    synchronize(issue_generator, config, main_section, dry_run)
  File "/home/fbence/.pyenv/versions/3.11.1/lib/python3.11/site-packages/bugwarrior/db.py", line 357, in synchronize
    for issue in issue_generator:
  File "/home/fbence/.pyenv/versions/3.11.1/lib/python3.11/site-packages/bugwarrior/services/__init__.py", line 572, in aggregate_issues
    raise RuntimeError(
RuntimeError: critical error in target 'trello'

Side question: it seems oracle is not supported for the api key? Although I'm guessing it's actually safe to make it public?

Is it possible to ask bugwarrior to be more verbose during pull?

ryneeverett commented 10 months ago

I'm a bit stumped on the main issue. Looking at the api, it sure seems like memberCreator is a guaranteed field: https://developer.atlassian.com/cloud/trello/rest/api-group-cards/#api-cards-id-actions-get.

Side question: it seems oracle is not supported for the api key? Although I'm guessing it's actually safe to make it public?

Correct. From Atlassian:

Because of the way the authorization flow works, the API key is intended to be publicly accessible. An API key by itself doesn't grant access to a user's Trello data. However, because API tokens grant access to the user's data, they should be kept secret.

Is it possible to ask bugwarrior to be more verbose during pull?

Yes, see log.level in the main configuration section. For example you could set log.level = DEBUG.

ferdinandyb commented 10 months ago

I ran it with debug, it definitely dies after a successful GET. I think it's safe to share, but since it's not my card I redacted some of the info:

[
  {
    "id": "5c62d2291c52686e642c1d2d",
    "idMemberCreator": "563740f5f2686c556323069c",
    "data": {
      "list": {
        "name": "Alkalmazások - ötletek",
        "id": "5bfe825a8242bb71a214a89e"
      },
      "board": {
        "shortLink": "I9N6VgPQ",
        "name": "Projektcsapatok és eredmények",
        "id": "5bfe81f9fac48b0308495558"
      },
      "card": {
        "shortLink": "kQE8Qc5Q",
        "idShort": 43,
        "name": "[redacted]",
        "id": "5c5d4febc09f542496b64c3e"
      },
      "text": "[redacted]"
    },
    "appCreator": null,
    "type": "commentCard",
    "date": "2019-02-12T14:03:21.595Z",
    "limits": {
      "reactions": {
        "perAction": {
          "status": "ok",
          "disableAt": 900,
          "warnAt": 720
        },
        "uniquePerAction": {
          "status": "ok",
          "disableAt": 17,
          "warnAt": 14
        }
      }
    }
  }
]

a quick solution for me is to just exclude this, as I don't need it, but it seems there is a type of card that does not have a memberCreator only an idMemberCreator.

ryneeverett commented 10 months ago

Thanks for the debug info, that's very helpful!

...it seems there is a type of card that does not have a memberCreator only an idMemberCreator.

I'm starting to think this may be a bug in the Trello api. My reading of the api docs is that "type": "commentCard" (as seen in your debug log) should always have a memberCreator field. That's the assumption that our code has and it doesn't seem to have been a problem until now.

ryneeverett commented 10 months ago

Ok, just found this thread, suggesting that this is what happens when the user who created the comment is deleted. Is that the case here?

If so, we should add a condition to handle this -- maybe substitute <deleted user> for the name.

ferdinandyb commented 10 months ago

Thanks for looking into this!

Indeed, this is the case (it says deleted user profile): image

So yeah, you probably want to guard against that. And I'll just exclude the board for now :)

ferdinandyb commented 10 months ago

Damn, there's no exclude_board option :(