j6k4m8 / goosepaper

generate and deliver a daily newspaper to you or your remarkable tablet
Apache License 2.0
271 stars 23 forks source link

Content ignored for RSS feed #80

Open captn3m0 opened 1 year ago

captn3m0 commented 1 year ago

I have the feed hosted locally, but here's the RSS feed:

beatroot.atom.zip

The output isn't what I'd expect:

image

j6k4m8 commented 1 year ago

I have a feeling this may be fixed in #82; let me know if you experience the issue still?

j6k4m8 commented 4 months ago

Hey @captn3m0 — still experiencing this, or can we close this issue? Happy to brainstorm if it's still happening!

captn3m0 commented 4 months ago

Still broken, but I have a guess on why. The code relies on summary, while the Atom feed I have publishes a content element instead.

I'm also unclear whether Goosepaper supports the RSS or the Atom standard, or both. feedparser seems to support normalization all the standards, so perhaps that should be used: https://feedparser.readthedocs.io/en/latest/content-normalization.html

I tried to parse the feed manually with feedparser, and it returns decent HTML, no matter which method I use:

e.summary
'<div><img src="https://beatrootnews.com/uploads/2023/06-June/05-Mon/sm/Antilla-building_647dc0f96050d.png" width="350" /></div>The Supreme Court today granted three weeks interim bail to former police officer Pradeep Sharma, who was arrested in connection with the Antilia bomb scare case and the killing of businessman Mansukh Hiran.<br /><br />A vacation bench of Justices Aniruddha Bose and Rajesh Bindal granted relief to Sharma after noting that he had sought interim bail on humanitarian grounds since his wife had developed serious complications after a surgery.On February 25, 2021, an explosives-laden SUV was found near Ambani\'s residence \'Antilia\' in south Mumbai. Businessman Hiran, who was in possession of the SUV, was found dead in a creek in neighbouring Thane on March 5, 2021.<br /><br />The allegation against Sharma, who belonged to the Mumbai Police\'s encounter squad that killed over 300 criminals in numerous encounters, was that he had helped his former colleague Waze in eliminating Hiran.<br /><br />Sharma was arrested in the case in June 2021 and is currently in judicial custody.'
>>> e['description']
'<div><img src="https://beatrootnews.com/uploads/2023/06-June/05-Mon/sm/Antilla-building_647dc0f96050d.png" width="350" /></div>The Supreme Court today granted three weeks interim bail to former police officer Pradeep Sharma, who was arrested in connection with the Antilia bomb scare case and the killing of businessman Mansukh Hiran.<br /><br />A vacation bench of Justices Aniruddha Bose and Rajesh Bindal granted relief to Sharma after noting that he had sought interim bail on humanitarian grounds since his wife had developed serious complications after a surgery.On February 25, 2021, an explosives-laden SUV was found near Ambani\'s residence \'Antilia\' in south Mumbai. Businessman Hiran, who was in possession of the SUV, was found dead in a creek in neighbouring Thane on March 5, 2021.<br /><br />The allegation against Sharma, who belonged to the Mumbai Police\'s encounter squad that killed over 300 criminals in numerous encounters, was that he had helped his former colleague Waze in eliminating Hiran.<br /><br />Sharma was arrested in the case in June 2021 and is currently in judicial custody.'
>>> e['summary']
'<div><img src="https://beatrootnews.com/uploads/2023/06-June/05-Mon/sm/Antilla-building_647dc0f96050d.png" width="350" /></div>The Supreme Court today granted three weeks interim bail to former police officer Pradeep Sharma, who was arrested in connection with the Antilia bomb scare case and the killing of businessman Mansukh Hiran.<br /><br />A vacation bench of Justices Aniruddha Bose and Rajesh Bindal granted relief to Sharma after noting that he had sought interim bail on humanitarian grounds since his wife had developed serious complications after a surgery.On February 25, 2021, an explosives-laden SUV was found near Ambani\'s residence \'Antilia\' in south Mumbai. Businessman Hiran, who was in possession of the SUV, was found dead in a creek in neighbouring Thane on March 5, 2021.<br /><br />The allegation against Sharma, who belonged to the Mumbai Police\'s encounter squad that killed over 300 criminals in numerous encounters, was that he had helped his former colleague Waze in eliminating Hiran.<br /><br />Sharma was arrested in the case in June 2021 and is currently in judicial custody.'

I tested against both 0.6.0 and 0.7.1. I also attempted to run it against other formats (Json/Mrss/SFeed), and while MRSS was parsed - it resulted in the same output.

Here's the generations against both 0.6.0 and 0.7.1:

j6k4m8 commented 4 months ago

Ah, thank you @captn3m0 for the thorough report! Feedparser looks like the right answer here, I wonder why we didn't originally do that...

Kernald commented 1 month ago

An easy public reproducer is xkcd's feed - no content shows up either.

j6k4m8 commented 1 month ago

Thanks for the report @Kernald! I'll address this as soon as I can get a few hours to tinker :)