search

academictorrents / academictorrents-docs

https://academictorrents.com/docs
16 stars 3 forks source link

Add Atom feed or h-entry item subscription option for tag search results #27

Closed thadguidry closed 2 years ago

thadguidry commented 2 years ago

Proposal

I'd like to subscribe to an Atom feed for datasets &c6=1 with tag wikidata . Given: https://academictorrents.com/browse.php?search=wikidata&c6=1 rather than remembering to do a search to check on new files of interesting tags.

Quick and Easy Alternatives:

Having the search results table formatted with h-entry item per file. http://microformats.org/wiki/h-feed where the h-feed element would be absent since it could be implied.

Additional Context

https://developer.mozilla.org/en-US/docs/Web/HTML/microformats

ieee8023 commented 2 years ago

Hmm, we have an RSS feed for all new public data https://academictorrents.com/rss.xml and also a feed for every public torrent with the title of the entry that you could filter: https://academictorrents.com/database.xml

One issue with creating dynamic search feeds is there is a community of people that latch on to these for meta torrent search platforms and we get so much traffic that it is like a DDOS to the site.

Another solution is if someone curated all the wikidata entries into a collection which has RSS and CSV feeds for automatic mirroring? https://academictorrents.com/collections.php

We also prevent the browse.php endpoint from being scraped directly so I don't think the h-entry solution would work. Maybe there is another solution I am not thinking of?

thadguidry commented 2 years ago

@ieee8023 Oh great then! That RSS feed works for me just fine.

ieee8023 commented 2 years ago

Oh, great idea! Will do! I'll leave this issue open until it is live.

thadguidry commented 2 years ago

@ieee8023 Hahaha, OK, it would be useful, and would indeed bring more searchability and visibility to AT through Search engines and even folks using Common Crawl.

ieee8023 commented 2 years ago

@thadguidry I just added those to the footer and reworked the feeds so they have extra info and cleanly present the description. Let me know if have any ideas for improvement. Thanks for bringing up this issue!

thadguidry commented 2 years ago

@ieee8023 Looks and works perfect! (tested with Feedly and Mozilla's Thunderbird)