search

RSS-Bridge / rss-bridge

The RSS feed for websites missing it
https://rss-bridge.org/bridge01/
The Unlicense
7.01k stars 1.02k forks source link

Bridge request for clean curl-impersonate of extant RSS feed #3767

Open Nitrousoxide opened 8 months ago

Nitrousoxide commented 8 months ago

Bridge request

Sometimes RSS feeds get broken by cloudfire protections Example: https://porforever.artstation.com/rss

Which will (probably) give a cloudflare error for you if you curl in the terminal, but will load in a browser just fine. Run this through a curl-impersonate container and it will come out fine however.

The existing "Feed reducer" bridge does generate a valid feed of this rss feed, bypassing the cloudlfare block. However it strips out the images and other formatting from the feed.

General information

No formatting changes from the underlying rss feed. Just a simple pass through the curl-impersonate to bypass cloudflare or other blocks.

No parameters needed other than a simple url to pass through curl-impersonate

dvikan commented 8 months ago

hello friend. instructions unclear.

but yes feeds are mangled when they pass through e.g. FeedReducer. this done by rssbridge itself.

Nitrousoxide commented 8 months ago

hello friend. instructions unclear.

but yes feeds are mangled when they pass through e.g. FeedReducer. this done by rssbridge itself.

I'm just looking for a bridge that passes an existing rss feed on the internet through curl-impersonate to bypass cloudflare protection without mangling it like feedreducer does.

Like how doing this will just spit out the unaltered rss feed.

podman run --rm lwthiker/curl-impersonate:0.5-chrome curl_chrome101 https://porforever.artstation.com/rss

Looking at the dockerfile for this project it looks like its built on curl-impersonate

FROM lwthiker/curl-impersonate:0.5-ff-slim-buster AS curlimpersonate

So just having a bridge to run existing rss feeds on the internet through it is useful if cloudflare is blocking them, like it is for artstation ones.

dvikan commented 8 months ago

FilterBridge is closest what you are looking for (if deployed with docker which uses curl-impersonate).

unfortunately rssbridge mangles the feed. it's an issue being worked on

dvikan commented 7 months ago

perhaps a new ProxyBridge can be introduced here. which would essentially be an http proxy for a feed.

dvikan commented 6 months ago

the original problem is kinda silly because rss feeds are supposed to be fetchable by bots

dvikan commented 5 months ago

recent commits may have partially fixed this problem. more of the feed is preserved.