search

thedod / feed2twister

Simple cron script to post RSS/ATOM items to Twister
GNU General Public License v3.0
10 stars 4 forks source link

utf8 support #10

Closed vladare closed 9 years ago

vladare commented 9 years ago

Hi, Is it possible to add support for utf8 feeds?

thedod commented 9 years ago

Can you give an example of such a feed so that I can test the code with it?

RealVegOs commented 9 years ago

Perhaps "my" Japanese feed has that problem too. Feed address is http://japanfocus.org/admin/site_manage/rss/recent.rss - The feed contains kanjis.

thedod commented 9 years ago

It looks like utf-8 (not some exotic encoding), so theoretically it should work. Don't know when I'll have time to test this (next month is kinda dense), but maybe you can paste the errors you're getting into some pastebin I'll get a clue about what's going wrong with that feed.

RealVegOs commented 9 years ago

I can paste that little output here:

blurp@blurper1:~$ /home/blurp/feed2twister/feed2twister.py -c /home/blurp/feed2twister-conf/japanfocus.conf INFO:root:http://japanfocus.org/admin/site_manage/rss/recent.rss Traceback (most recent call last): File "/home/blurp/feed2twister/feed2twister.py", line 180, in main(n) File "/home/blurp/feed2twister/feed2twister.py", line 105, in main eid = '{0}|{1}'.format(feed_url,e.id) File "/usr/lib/python2.7/dist-packages/feedparser.py", line 424, in getattr raise AttributeError, "object has no attribute '%s'" % key AttributeError: object has no attribute 'id'

(Of course I'm not blurp and my computer is not blurper1) It works with other feeds. I tested with a shitty Twitter profile parsing feed.

RealVegOs commented 9 years ago

Sorry for opening again: First line look good but still ends with attribute error: (Same result after deleting the db file)

/home/blurb/feed2twister/feed2twister.py -c /home/blurb/feed2twister-conf/japanfocus.conf INFO:root:http://japanfocus.org/admin/site_manage/rss/recent.rss INFO:root:will post http://ur1.ca/nbikv 'Comfort Women' Denial and the Japanese Right INFO:root:will post http://ur1.ca/nbikw âSomething Dreadful Happened in the Pastâs Local Peace Museums: The Conversion of Peace Osaka INFO:root:will post http://ur1.ca/nbiky An â as Japanâs Grand Strategy: New Dynamism or Dead-End? INFO:root:will post http://ur1.ca/nbikz To Whom Does the Sea Belong? Questions Posed by the Henoko Assessment WARNING:root:Quota reached. Skipping 15 items: Traceback (most recent call last): File "/home/blurb/feed2twister/feed2twister.py", line 183, in main(n) File "/home/blurb/feed2twister/feed2twister.py", line 148, in main eeid = '{0}|{1}'.format(feed_url, ee.id) File "/usr/lib/python2.7/dist-packages/feedparser.py", line 424, in getattr raise AttributeError, "object has no attribute '%s'" % key AttributeError: object has no attribute 'id'

Perhaps a shitty RSS feed...

thedod commented 9 years ago

Reopening the ticket — @RealVegOs seems to still have issues with japanfocus

thedod commented 9 years ago

My mistake :confounded: