Deal with unicode warning.

gt-big-data / QDoc

Quick & Dirty Operating Crawler

4 stars 1 forks source link

Deal with unicode warning. #38

Closed supersam654 closed 8 years ago

supersam654 commented 8 years ago

When crawling some articles, something complains about

/home/bdc/anaconda2/lib/python2.7/difflib.py:433: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  a[besti-1] == b[bestj-1]:

In general, article objects should expose everything as unicode strings (and feed objects should expose everything as unicode strings).

supersam654 commented 8 years ago

Should be fixed now. I'll open a new issue if I see the problem pop up again (I can't reproduce it locally any more).