MetricsGrimoire / Bicho

Bicho is a command line based tool used to parse bug/issue tracking systems
http://metricsgrimoire.github.com/Bicho/
GNU General Public License v2.0
71 stars 31 forks source link

Some Bugzilla Issues cannot be parsed #165

Open simplay opened 7 years ago

simplay commented 7 years ago

I am using the current Bicho Master (0a91890a9cb59df9560cf3d4563031db94197afa ).

I Already have used bicho successfully by fetching all issues labeled WONTFIX. (more precisely: https://bugzilla.mozilla.org/buglist.cgi?quicksearch=WONTFIX product:"Firefox" &list_id=13425305)

I tried the same for issues labeled as FIXED. Soon after I fetched several issues, bicho crashed, returning with an exception TypeError: 'NoneType' object is not callable. I identified the issue that causes this exception (this one: https://bugzilla.mozilla.org/show_bug.cgi?id=282502).

I can reproduce this behaviour by either attempting to fetch all FIXED issues or by invoking

$ bicho --db-user-out=root --db-password-out="MY_DB_PW" --db-database-out=DATABASE_NAME --db-driver-out=mysql --db-hostname-out=localhost --db-port-out=3306 --backend-user="MY_EMAIL_ADDRESS" --backend-password="MY_BACKEND_PW" -d 1 -b bg -u "https://bugzilla.mozilla.org/show_bug.cgi?id=282502"

In the following the corresponding stack trace:

Checking URL: https://bugzilla.mozilla.org
Running Bicho with delay of 1 seconds
Logged in bugzilla as eckert.remo@gmail.com
/usr/local/lib/python2.7/dist-packages/storm-0.20-py2.7-linux-x86_64.egg/storm/database.py:454: Warning: Data truncated for column 'name' at row 1
return function(*args, **kwargs)
Traceback (most recent call last):
File "/usr/local/bin/bicho", line 25, in <module>
retval = bicho.main.main()
File "/usr/local/lib/python2.7/dist-packages/bicho/main.py", line 56, in main
backend.run()
File "/usr/local/lib/python2.7/dist-packages/bicho/backends/bg.py", line 952, in run
self._process_issues()
File "/usr/local/lib/python2.7/dist-packages/bicho/backends/bg.py", line 1048, in _process_issues
self._retrieve_issues(ids, url, self.tracker.id)
File "/usr/local/lib/python2.7/dist-packages/bicho/backends/bg.py", line 1107, in _retrieve_issues
changes = self._retrieve_issue_activity(base_url, issue.issue)
File "/usr/local/lib/python2.7/dist-packages/bicho/backends/bg.py", line 1124, in _retrieve_issue_activity
changes = parser.parse_changes()
File "/usr/local/lib/python2.7/dist-packages/bicho/backends/bg.py", line 326, in parse_changes
removed = unicode(cols[3].contents[0].strip())
TypeError: 'NoneType' object is not callable

To prevent the crash I modified bg.py as follows:

diff --git a/bicho/backends/bg.py b/bicho/backends/bg.py
index 84bed88..989e52e 100644
--- a/bicho/backends/bg.py
+++ b/bicho/backends/bg.py
@@ -323,7 +323,10 @@ class SoupHtmlParser():
                     field = unicode(aux_c.replace("\n", "").strip())
                 else:
                     field = unicode(cols[2].contents[0].replace("\n", "").strip())
-                removed = unicode(cols[3].contents[0].strip())
+                try:
+                    removed = unicode(cols[3].contents[0].strip())
+                except TypeError as detail:
+                    print 'Handling run-time error:', detail
                 added = unicode(cols[4].contents[0].strip())
             else:
                 # same as above with the Attachment example

So, I basically skip such problem causing issues, which is certainly not a propper fix. Please note that most of the issues having the fixed label are working fine, just a few are causing this problem.

Is there anything I am doing wrong (did I miss to pass additional parameters)? Hope this helps to identify the problem.