newslynx / opportunities

A garden of NewsLynx futures
1 stars 0 forks source link

Twitter list to event search terms isn't ASCII? #175

Closed mhkeller closed 8 years ago

mhkeller commented 8 years ago

search keyword is 'governing'.

Possibly related: http://stackoverflow.com/questions/10934184/python-unicodedecodeerror-ascii-codec-cant-decode-byte-0xe2-ordinal-not-in-r

Traceback (most recent call last):
  File \"/usr/local/lib/python2.7/dist-packages/newslynx/merlynne.py\", line 123, in run
    data = sous_chef.cook()
  File \"/opt/newslynx/newslynx-core/newslynx/sc/__init__.py\", line 126, in cook
    raise SousChefExecError(format_exc())
    SousChefExecError: Traceback (most recent call last):
  File \"/opt/newslynx/newslynx-core/newslynx/sc/__init__.py\", line 122, in cook
    data = self.load(data)  # load the data.
  File \"/root/.newslynx/sous-chefs/newslynx-sc-twitter/newslynx_sc_twitter/events.py\", line 136, in load
    for d in data:
  File \"/opt/newslynx/newslynx-core/newslynx/sc/__init__.py\", line 80, in serialize
    for item in data:
  File \"/root/.newslynx/sous-chefs/newslynx-sc-twitter/newslynx_sc_twitter/events.py\", line 126, in run
    tweet = self.filter(tweet)
  File \"/root/.newslynx/sous-chefs/newslynx-sc-twitter/newslynx_sc_twitter/events.py\", line 57, in filter
    m = self.options['search_query'].match(to_search)
  File \"/usr/local/lib/python2.7/dist-packages/newslynx/lib/search.py\", line 186, in match
    tests.append(self._simple_match(term['term'], t, raw))
  File \"/usr/local/lib/python2.7/dist-packages/newslynx/lib/search.py\", line 201, in _simple_match
    elif term in raw.lower():
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 90: ordinal not in range(128)
mhkeller commented 8 years ago

Or it could be tweet text that uses special characters

abelsonlive commented 8 years ago

yeah i just need to encode that string before making the comparison.