ksh-b / raven

News Aggregator
GNU General Public License v3.0
242 stars 4 forks source link

how to add custom sources? #4

Closed akashpal-21 closed 6 months ago

akashpal-21 commented 7 months ago

Currently for India only two channels could be selected -; how do we add more - sources too little.

SleepDaemon commented 7 months ago

Currently for India only two channels could be selected -; how do we add more - sources too little.

There isn't a way to add custom sources

ksh-b commented 7 months ago

You can request sources. I could add them in future releases.

akashpal-21 commented 7 months ago

You can request sources. I could add them in future releases.

For India - def scraping The Hindu or Indian Express would be good !

Please consider if possible :))

zinhuofc commented 7 months ago

You can request sources. I could add them in future releases.

Can't you implement feed search by name/site and have the option for the person to add it manually?

ksh-b commented 7 months ago

You can request sources. I could add them in future releases.

Can't you implement feed search by name/site and have the option for the person to add it manually?

Each site needs to be scraped differently, so I don't think its possible to have that option. RSS feeds is an alternative, but not all sites support it. There are already apps that support RSS feeds hence I have no plans of adding it.

zinhuofc commented 7 months ago

You can request sources. I could add them in future releases.

Can't you implement feed search by name/site and have the option for the person to add it manually?

Each site needs to be scraped differently, so I don't think its possible to have that option. RSS feeds is an alternative, but not all sites support it. There are already apps that support RSS feeds hence I have no plans of adding it.

Ah yes, I had misunderstood, that the app was an RSS feed reader, the only problem for me is that these readers do not support automatic translation

can you add then please. https://www.androidpolice.com/ https://www.xda-developers.com/ www.neowin.net https://www.ghacks.net/ https://www.pcmag.com/ https://www.gamevicio.com/ https://www.adrenaline.com.br/noticias/ {brazil}

ksh-b commented 7 months ago

You can request sources. I could add them in future releases.

For India - def scraping The Hindu or Indian Express would be good !

Please consider if possible :))

Added in v0.9.0

D3SOX commented 6 months ago

Since this thread didn't help me answer my question I'm gonna post it here: Is custom feed support something you want to support in the future? Like from a RSS feed URL or importing OPML files

ksh-b commented 6 months ago

@D3SOX Thanks for the question. TLDR: Yes, custom feeds might be supported in the future.


When this issue was created I didn't have a plan to support custom feeds as for extracting the full article - scraping from the source would still be needed (RSS feeds dont have the whole content).

But a few days ago I came across smort.io which could help extract whole content. That would mean I just have to scrape smort and not the individual custom sources.

However I think depending only on smort might cause issues (it may go down etc). So I am planning to add a fallback first in case that happens, after which custom sources could be added.

Hope it makes sense // Let me know if you any other ideas.

D3SOX commented 6 months ago

@ksh-b Thanks for the detailed insights. I think supporting custom feeds without the full content is okay at first when there's a button to open it in the browser or an In-App-WebView.

The previous RSS readers I've used have basic support for fetching full contents locally:

Having something like smort.io as fallback would be a great addition.

ksh-b commented 6 months ago

Rss/Atom support added in v0.12.0

Subscriptions >custom > RSS

D3SOX commented 6 months ago

Great! Should I open a separate issue for OPML import/export support?

D3SOX commented 6 months ago

And I can't seem to add feed URLs because it seems to load indefinitely

Screenshot_20240515-184312_cropped

ksh-b commented 6 months ago

Yes. You can open separate issues for both. The feed url in your screenshot doesnt load for me as well. OPML support might take time.

ksh-b commented 6 months ago

http://pointieststick.com/feed works with morss though

mr-gosh commented 6 months ago

There are sites like https://www.haz.de/ which release every article for a few hours and after that period they are out behind a paywall.

There the scraper would be awesome to scrape the whole article.

And I would like to see heise.de as the biggest German it site in the supported sources too.

Achyuth072 commented 5 months ago

Scraping 'Times Of India' would be nice.