search

cfpb / github-wiki-search

:warning: DEPRECATED :warning:
Creative Commons Zero v1.0 Universal
7 stars 17 forks source link

Error indexing Jira issues #112

Closed OrlandoSoto closed 6 years ago

OrlandoSoto commented 6 years ago

Traceback (most recent call last): File "server/index.py", line 16, in gh_repos, ghe_repos, jira_users, jira_projs = search.index() File "/home/ec2-user/github-wiki-search/server/search/init.py", line 23, in index jira_users, jira_projs = jira.index() File "/home/ec2-user/github-wiki-search/server/search/jira.py", line 48, in index 'author': issue['fields']['creator']['name'], TypeError: 'NoneType' object is unsubscriptable Exception KeyError: KeyError(140382821633072,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored

OrlandoSoto commented 6 years ago

steps to reproduce:

ssh ec2-user@10.153.97.166

cd /home/ec2-user/github-wiki-search

python server/index.py

the index script will run for several minutes and then will report the above error

OrlandoSoto commented 6 years ago

After some local debugging, it appears to me that there are some Jira issues that contain missing/corrupt fields (create/name). This may be causing the indexing script to fail. I commented out these lines

OrlandoSoto commented 6 years ago

After I commented out the above lines and re-ran the script, another error surfaced:

Jira: 92178 issues and comments (273.0 secs)
Traceback (most recent call last):
  File "server/index.py", line 17, in <module>
    autocomplete.index(gh_repos, ghe_repos, jira_users, jira_projs)
  File "/home/ec2-user/github-wiki-search/server/autocomplete/__init__.py", line 31, in index
    users = set(jira_users).union(gh_users.get())
  File "/home/ec2-user/github-wiki-search/server/autocomplete/gh_users.py", line 19, in get
    return set((user['login'] for user in itertools.chain(*user_iters)))
  File "/home/ec2-user/github-wiki-search/server/autocomplete/gh_users.py", line 19, in <genexpr>
    return set((user['login'] for user in itertools.chain(*user_iters)))
  File "/usr/lib/python2.6/site-packages/server/utils.py", line 12, in iter_get
    resp = client.get()
  File "/usr/lib/python2.6/site-packages/universalclient/universalclient.py", line 68, in __call__
    return self._setArgs(method=calledAttr.lower()).request(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/universalclient/universalclient.py", line 70, in __call__
    return _getattr(calledAttr)(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/universalclient/universalclient.py", line 108, in request
    return requests.request(**attributes)
  File "/usr/lib/python2.6/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 456, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.6/site-packages/requests/adapters.py", line 382, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:492: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
Exception KeyError: KeyError(140132217826512,) in <module 'threading' from '/usr/lib64/python2.6/threading.pyc'> ignored
OrlandoSoto commented 6 years ago

It seems to fail at SSL certificate verification on the search.demo server though I was able to run this in my local environment.

OrlandoSoto commented 6 years ago

The SSL issue has been resolved. It turned out to be a problem with the docker-machine I created. I used a different docker-machine and ran without SSL issues.

OrlandoSoto commented 6 years ago

The Jira indexing issue was fixed in this commit: https://github.com/cfpb/github-wiki-search/commit/0460a6ed99f42a7505bb47e1e7dd2fc6acc0d068