XPathBridge not working as expected

RSS-Bridge / rss-bridge

The RSS feed for websites missing it

https://rss-bridge.org/bridge01/

The Unlicense

7k stars 1.02k forks source link

XPathBridge not working as expected #3951

Closed clintiepoo closed 4 months ago

clintiepoo commented 4 months ago

Describe the bug XPathBridge is not parsing this site as expected. It could be user error.

To Reproduce (on my self hosted agent as well as on the public one: https://rss-bridge.org/bridge01/)

Use XPathBridge
Enter web page URL: https://substack.com/browse/staff-picks
Item selector: //div[contains(@class, 'reader2-post-container')]
Item Title Selector: .//div[contains(@class,'reader2-post-title')]

No feed is generated.

Expected behavior Feed should be created with this input. Other fields (url, description) can be added later.

Screenshots

Additional context These same expressions work in the browser console (Firefox).

tillcash commented 4 months ago

When attempting to access the webpage (https://substack.com/browse/staff-picks) using curl, the content wasn't visible. It appears that the site relies on JavaScript, as it fetches data from https://substack.com/api/v1/reader/posts/staff-picks.

Since RSS-Bridge doesn't handle JavaScript, this likely explains why the content isn't being parsed correctly and why the XPathBridge module fails to generate a feed.

clintiepoo commented 4 months ago

When attempting to access the webpage (https://substack.com/browse/staff-picks) using curl, the content wasn't visible. It appears that the site relies on JavaScript, as it fetches data from https://substack.com/api/v1/reader/posts/staff-picks.

Since RSS-Bridge doesn't handle JavaScript, this likely explains why the content isn't being parsed correctly and why the XPathBridge module fails to generate a feed.

Thanks - I understand it now and will close the issue.