github import fails

[visibilityspots]

Hi,

Trying to get my issues (username: visibilityspots) into taskwarrior setup with your bugwarrior script is failing:

2013-09-20T20:26:13Z:WARNING:config:URLs will not be shortened with bit.ly 2013-09-20T20:26:13Z:WARNING:config:URLs will not be shortened with bit.ly 2013-09-20T20:26:13Z:INFO:bugwarrior:Starting to aggregate remote issues. 2013-09-20T20:26:13Z:INFO:github:Working on [github] 2013-09-20T20:26:17Z:DEBUG:github: Found 12 total. 2013-09-20T20:26:17Z:DEBUG:github: Pruned down to 0 2013-09-20T20:26:17Z:INFO:github:Done with [github] 2013-09-20T20:26:17Z:INFO:bugwarrior:Done aggregating remote issues. 2013-09-20T20:26:17Z:INFO:db:completed=0:new=0:

And I don't see any of them appearing in taks list..

[ralphbean]

Hm, it is saying Pruned down to 0 which means it found 12 issues, but discarded them because it thought they were already in your task list.

[ralphbean]

Can you try doing a grep in your ~/.task/ db to see if they're in there?

Failing that, can you paste your ~/.bugwarriorrc file here (obviously, with any passwords removed first).

[visibilityspots]

Hi,

the grep command on php didn't showed the issue from github. My .bugwarriorrc file:

# General stuff.
[general]
# Here you define a comma separated list of targets.  Each of them must have a
# section below determining their properties, how to query them, etc.  The name
# is just a symbol, and doesn't have any functional importance.
targets = my_github

# log.level specifices the verbosity.  The default is DEBUG.
# log.level can be one of DEBUG, INFO, WARNING, ERROR, CRITICAL, DISABLED
log.level = DEBUG

# If log.file is specified, output will be redirected there.  If it remains
# unspecified, output is sent to sys.stderr
log.file = ~/.task/bugwarrior.log

# The bitly username and api key are used to shorten URLs to the issues for your
# task list.  If you leave these options commented out, then the full URLs
# will be used in your task list.
#bitly.api_user = YOUR_USERNAME
#bitly.api_key = YOUR_API_KEY

# This is an experimental mode where bugwarrior will query all of your
# online sources simultaneously.  It works as far as I've tested it, so
# give it a try.  Please backup your ~/.task/ directory first and report
# any problems if you encounter them at
# https://github.com/ralphbean/bugwarrior/issues
multiprocessing = False

# Configure the default description or annotation length.
#annotation_length = 45
#description_length = 35

# This section is for configuring notifications when bugwarrior-pull runs,
# and when issues are created, updated, or deleted by bugwarrior-pull.
# Three backend are currently suported:
#
#  - growlnotify (v2)   Mac OS X   "gntp" must be installed
#  - gobject            Linux      python gobject must be installed
#  - pynotify           Linux      "pynotify" must be installed
#
# To configure, adjust the settings below.  Note that neither of the
# "sticky" options have any effect on Linux with pynotify.  They only work
# for growlnotify.
[notifications]
notifications = False
# backend = growlnotify
# finished_querying_sticky = False
# task_crud_sticky = True

# This is a github example.  It says, "scrape every issue from every repository
# on http://github.com/ralphbean.  It doesn't matter if ralphbean owns the issue
# or not."
[my_github]
service = github
username = visibilityspots
default_priority = M

# I want taskwarrior to include issues from all my repos, except these
# two because they're spammy or something.
# exclude_repos =

# Note that login and username can be different.  I can login as me, but
# scrape issues from an organization's repos.
login = ##@##
passw = XXX

[ralphbean]

Ah, can you try specifying your username as just visibilityspots instead of SOME-EMAIL@DOMAIN.COM? That's my first guess. If that doesn't work, I'll pull down your .bugwarriorrc and try to see if I can duplicate the issue on my machine.

[ralphbean]

Ah, can you try specifying your username as just visibilityspots

Sorry, I meant to say "set your login to visibilityspots.

[visibilityspots]

Changed my login to visibilityspots instead of my email address but that also gave the same debugging and didn't imported the issues.

I tested the credentials to be sure and they are working fine..

[my_github]
service = github
username = visibilityspots
default_priority = M

# I want taskwarrior to include issues from all my repos, except these
# two because they're spammy or something.
# exclude_repos =

# Note that login and username can be different.  I can login as me, bui
# scrape issues from an organization's repos.
login = visibilityspots
passw = xx

[ralphbean]

Weird. You can see the log statements about "finding" and "pruning" here -> https://github.com/ralphbean/bugwarrior/blob/develop/bugwarrior/services/github.py#L78

Inbetween, it makes a series of calls to this include method to determine what issues should be dropped -> https://github.com/ralphbean/bugwarrior/blob/develop/bugwarrior/services/__init__.py#L56

For some reason, it is deciding to drop all of your issues there. Perhaps you accidentally have only_if_assigned set somewhere?

[visibilityspots]

Hi,

I searched my config for the only_if_assigned option but it's commented everywhere in the file. I did a test using the option with my username but that also gave the same result.

It's a long time ago I dived into python, I just added 2 debugging lines into github.py on line 61:

print "DEBUGGING:", len(issues) 
issues = filter(self.include, issues)
print "DEBUGGING:", len(issues)

And the output is this:

bugwarrior-pull 
DEBUGGING: 12
DEBUGGING: 0

I also printed out the array, before the filter option I get all the issues, after the filter it's emtpy..

[ralphbean]

sigh - Sorry you've had to go through this.

Try adding debug statements also to the include method in bugwarrior/services/__init__.py so that it looks like:

def include(self, issue):
    """ Return true if the issue in question should be included """

    # TODO -- evaluate cleaning this up.  It's the ugliest stretch of code
    # in here.

    only_if_assigned, also_unassigned = None, None
    try:
        only_if_assigned = self.config.get(
            self.target, 'only_if_assigned')
    except Exception:
        pass

    try:
        also_unassigned = self.config.getboolean(
            self.target, 'also_unassigned')
    except Exception:
        pass

    print "DEBUGGING:  title =", issue.get('title', None)
    print "DEBUGGING:  only_if_assigned =", onlyif_assigned
    print "DEBUGGING:  also_unassigned =", also_unassigned
    print "DEBUGGING:  owner =", self.get_owner(issue)

    if only_if_assigned and also_unassigned:
        return self.get_owner(issue) in [only_if_assigned, None]
    elif only_if_assigned and not also_unassigned:
        return self.get_owner(issue) in [only_if_assigned]
    elif not only_if_assigned and also_unassigned:
        return self.get_owner(issue) in [None]
    elif not only_if_assigned and not also_unassigned:
        return self.get_owner(issue) in [None]
    else:
        pass  # Impossible to get here.

[visibilityspots]

I really don't mind, if only I can help to make this tool better I'm happy. (And get it working too, looked for this piece of software for ages :-) )

if commenting out the first line, I don't get anything, once I comment it I get some output

        #print "DEBUGGING:  title =", issue.get('title', None)
        print "DEBUGGING:  only_if_assigned =", only_if_assigned
        print "DEBUGGING:  also_unassigned =", also_unassigned
        print "DEBUGGING:  owner =", self.get_owner(issue)

the output:

DEBUGGING:  only_if_assigned = None
DEBUGGING:  also_unassigned = None
DEBUGGING:  owner = {u'following_url': u'https://api.github.com/users/visibilityspots/following{/other_user}', u'events_url': u'https://api.github.com/users/visibilityspots/events{/privacy}', u'organizations_url': u'https://api.github.com/users/visibilityspots/orgs', u'url': u'https://api.github.com/users/visibilityspots', u'gists_url': u'https://api.github.com/users/visibilityspots/gists{/gist_id}', u'html_url': u'https://github.com/visibilityspots', u'subscriptions_url': u'https://api.github.com/users/visibilityspots/subscriptions', u'avatar_url': u'https://2.gravatar.com/avatar/8b995ac15f8b125905bc99f9e9190138?d=https%3A%2F%2Fidenticons.github.com%2F55dee46b54eef6313741e14e22091c88.png', u'repos_url': u'https://api.github.com/users/visibilityspots/repos', u'received_events_url': u'https://api.github.com/users/visibilityspots/received_events', u'gravatar_id': u'8b995ac15f8b125905bc99f9e9190138', u'starred_url': u'https://api.github.com/users/visibilityspots/starred{/owner}{/repo}', u'login': u'visibilityspots', u'type': u'User', u'id': XXX, u'followers_url': u'https://api.github.com/users/visibilityspots/followers'}

[ralphbean]

Ah, I see.. its because you have all your bugs actually assigned to yourself.

See here:

https://github.com/ralphbean/bugwarrior/blob/develop/bugwarrior/services/__init__.py#L81

The logic of the fourth conditional goes: "if only_if_assigned is None and also_unassigned is None, then only include the issue if it is assigned to no-one". Since your bug is actually assigned to you, that block returns False and the issue gets pruned.

But... that doesn't make sense! If you do not have only_if_assigned set, then you should get all bugs, assigned or not. I'll push a commit that should fix that -- can you try it out?