Duplicate issues stored to taskwarrior database.

[edigiacomo]

bugwarrior version: 1.4.0

I have multiple github services (one for my personal repos and one for each organization in which I am involved). The issues of my personal repos are duplicated.

My .bugwarriorrc:

[general]
targets = edigiacomo_github, arpae_simc_github, r_map_github
inline_links = False
annotation_links = False
annotation_comments = False
legacy_matching = False

[edigiacomo_github]
service = github
only_if_assigned = edigiacomo
github.username = edigiacomo
github.login = edigiacomo
github.password = ...
github.include_repos = django-statusboard, arkimet-tools
description_template = {{githubnumber}}: {{githubtitle}}
add_tags = github, edigiacomo_github

[arpae_simc_github]
service = github
only_if_assigned = edigiacomo
github.username = arpa-simc
github.login = edigiacomo
github.password = ...
github.include_repos = meteo-vm2, wreport, dballe, arkimet, simc-opendata-examples, arkimet-postprocessor-suite
description_template = {{githubnumber}}: {{githubtitle}}
add_tags = github, arpae_simc_github

[r_map_github]
service = github
only_if_assigned = edigiacomo
github.username = r-map
github.login = edigiacomo
github.password = ...
github.include_repos = rmap
description_template = {{githubnumber}}: {{githubtitle}}
add_tags = github, r_map_github

The first time that I run bugwarrior-pull, the issues of the repo edigiacomo/django-statusboard are repeated 3 times (one for each github service).

Playing with the code, I found that the "directly assigned issues" (GithubService.get_directly_assigned_issues) are always retrieved. Each process has its own copy of these issues (it seems that https://api.github.com/user/issues return issues from my personal repos only). When the program iterates over all the issues found, it doesn't detect that some issues are duplicated and so they are added multiple times.

Maybe am I missing something?

Thank you!

[ryneeverett]

Don't have time to look into this right now, but I would suggest trying our develop branch to see if we didn't (perhaps accidentally) fix this. I know we've made some changes to the way github issues are filtered since the last release.

[edigiacomo]

Thanks, with the develop branch and a proper configuration the duplication can be avoided in my specific case (thanks to the new filter implementation for github.include_repos).

TL;DR: I have wrote a fix to avoid that a new issue is added more than once, I'll send a PR.

I think that the problem still persists: when more than one Github service return the same new issue (e.g. the ones from Github.get_directly_assigned_issues), then that issue is marked more than once as a new issue and it's stored more than once.

E.g. removing github.include_repos, some issues are added multiple times:

INFO:bugwarrior.db:Adding 13 tasks
INFO:bugwarrior.db:Adding task 12: Datetime picker for incident time
INFO:bugwarrior.db:Adding task 10: Subscribe to service update
INFO:bugwarrior.db:Adding task 31: Configurable permissions
INFO:bugwarrior.db:Adding task 11: Documentation is missing
INFO:bugwarrior.db:Adding task 12: Datetime picker for incident time
INFO:bugwarrior.db:Adding task 11: Documentation is missing
INFO:bugwarrior.db:Adding task 31: Configurable permissions
INFO:bugwarrior.db:Adding task 10: Subscribe to service update
INFO:bugwarrior.db:Adding task 110: server: Borinud ArkimetBufrDB deve sapere cosa misura ogni stazione
INFO:bugwarrior.db:Adding task 122: server: remove CDN JS and CSS from Borinud HTML
INFO:bugwarrior.db:Adding task 113: server: implement ArkimetVm2DB.query_stations in Borinud
INFO:bugwarrior.db:Adding task 195: server: is possible an improvement for time regularization in dballe graphite finder?
INFO:bugwarrior.db:Adding task 197: server: add a dynamic menu for find data in web services

If I run bugwarrior-pull a second time, an exception for each duplicated issue is raised:

ERROR:bugwarrior.db:Multiple matches: Issue 10: Subscribe to service update matched multiple IDs: {UUID('eb7ba810-e635-4a05-a184-98284029f9a2'), UUID(
'8c141f61-92a3-4f9c-ae85-43d863cba666')}
Traceback (most recent call last):
  File "/home/edg/src/bugwarrior/e/lib/python3.5/site-packages/bugwarrior/db.py", line 330, in synchronize
    tw, key_list, issue, legacy_matching=legacy_matching
  File "/home/edg/src/bugwarrior/e/lib/python3.5/site-packages/bugwarrior/db.py", line 206, in find_local_uuid
    possibilities
bugwarrior.db.MultipleMatches: Issue 10: Subscribe to service update matched multiple IDs: {UUID('eb7ba810-e635-4a05-a184-98284029f9a2'), UUID('8c141f61-92a3-4f9c-ae85-43d863cba666')}

and the duplicated (open) issues are then closed (multiple times):

INFO:bugwarrior.services:Done aggregating remote issues.
INFO:bugwarrior.db:Adding 0 tasks
INFO:bugwarrior.db:Updating 0 tasks
INFO:bugwarrior.db:Closing 8 tasks
INFO:bugwarrior.db:Completing task 8d72447a-cb13-43ba-9b50-0a1ad406916b 31: Configurable permissions
INFO:bugwarrior.db:Completing task 9d541d56-4d95-43bd-88b7-33b3f18d98d1 11: Documentation is missing
INFO:bugwarrior.db:Completing task 075aff7e-f1a8-422c-b5e7-6b2592fbafbf 11: Documentation is missing
INFO:bugwarrior.db:Completing task eb7ba810-e635-4a05-a184-98284029f9a2 10: Subscribe to service update
INFO:bugwarrior.db:Completing task cf0d4936-d748-4cf2-a85f-527ad5047663 31: Configurable permissions
INFO:bugwarrior.db:Completing task 5ed9c6c1-d645-45bb-be24-cad613ca7898 12: Datetime picker for incident time
INFO:bugwarrior.db:Completing task 8c141f61-92a3-4f9c-ae85-43d863cba666 10: Subscribe to service update
INFO:bugwarrior.db:Completing task c29144f3-c854-4495-babb-d9dfa3a54b99 12: Datetime picker for incident time

Checking if an issue was already added in the same run could solve the issue.

[Mte90]

I got the same error with github right now.

[ryneeverett]

I believe this should have been resolved by #654.