[Feature] Some sites block scraping content without javascript.

[sherlcok314159]

Some sites can not be scraped without javascript. And I tried different useragents such as curl/8.21. All the useragents failed.

Site: https://rsshub.app/zhubai/posts/havefun

[Alkarex]

You can try with https://github.com/lwthiker/curl-impersonate/ , which sometimes help. Otherwise you will need a more sophisticated system.

[sherlcok314159]

Thanks. But how can I combine this with freshrss?

[Alkarex]

A typical way is to use a system such as RSS Bridge, which outputs an RSS feed, which can be consumed by FreshRSS. But first step is to find an approach that works manually.

[squromiv]

Some sites can not be scraped without javascript

Try feedless tool. It can help in some cases.

[sherlcok314159]

Thanks for the above replies. My solution is to use a local headless browser to handle this by python. It is quite light.