Open alextreppass opened 14 years ago
sorry for the lack of docs, was in a hurry to get my indexer up and going again.
I've updated the wiki with answers to your first question
You can't avoid crawling an entire newsgroup, even when you store just the titles/ID of a post containing an NZB. Usenet has no search capabilities. Ive built the crawler to be able to incrementally crawl after the initial index is built. I'm going to expand this further.
For the part of usenet that does not have nzb files, i wanted a search similar function but I can't determine a good way to make this work. Filenames seem too random to make pattern matching work well. If it could be made to work you could temporarily store titles, removing them if a matching nzb post is found, and condensing the rest into a list of post IDs. This would take some considerable computing power though.
Trying to get a handle on the sorts of things you're doing here.