search

btdig / dhtcrawler2

dhtcrawler is a DHT crawler written in erlang. It can join a DHT network and crawl many P2P torrents. The program save all torrent info into database and provide an http interface to search a torrent by a keyword
Other
1.32k stars 305 forks source link

New search results not showing on BTdig since 24hrs #10

Closed DarthAdriano closed 10 months ago

DarthAdriano commented 11 months ago

Since yesterday the latest search results for anything are 1-2 days old. When you click recent findings, you can see new magnets in search results are being uploaded every minute but nothing new shows when u directly search for it via age or relevance. Haven't experienced this glitch before.

jelockwood commented 10 months ago

@DarthAdriano

I can confirm I am seeing the same thing. The 'recent' page shows new magnet links of seconds ago but the search function does not return any entries since the problem started and hence nothing for now 4 days and counting.

ghost commented 10 months ago

Not a good sign. The last time i saw this sort of thing.. about a week later the page was gone. Then a few days later, the domain.

DarthAdriano commented 10 months ago

@DarthAdriano

I can confirm I am seeing the same thing. The 'recent' page shows new magnet links of seconds ago but the search function does not return any entries since the problem started and hence nothing for now 4 days and counting.

Yes 4 days and counting for me too. Nothing has changed and they are not replying address the issue via email either.

DarthAdriano commented 10 months ago

Not a good sign. The last time i saw this sort of thing.. about a week later the page was gone. Then a few days later, the domain.

what site did that happen with? Do we have any alternative like BTdig?

ghost commented 10 months ago

Not a good sign. The last time i saw this sort of thing.. about a week later the page was gone. Then a few days later, the domain.

what site did that happen with? Do we have any alternative like BTdig?

It was an ebook site. Details not too important now since its long gone.

Myself, not aware of any hosted DHT scrapers out there as i have never had to look. Just the stupid 'torrent sites' which A: cant trust most of them and B: cant trust the rest :) . ( ok exaggerating, but only sightly, as they are all a risk. May be time to run it ourselves. Perhaps others will know of something.

We also might get lucky and the admins are just busy due to holidays.

DarthAdriano commented 10 months ago

Not a good sign. The last time i saw this sort of thing.. about a week later the page was gone. Then a few days later, the domain.

what site did that happen with? Do we have any alternative like BTdig?

It was an ebook site. Details not too important now since its long gone.

Myself, not aware of any hosted DHT scrapers out there as i have never had to look. Just the stupid 'torrent sites' which A: cant trust most of them and B: cant trust the rest :) . ( ok exaggerating, but only sightly, as they are all a risk. May be time to run it ourselves. Perhaps others will know of something.

We also might get lucky and the admins are just busy due to holidays.

RARBG was sooo good. But love DHT search engines. I can only hope they are busy and will return but not even acknowledging the emails I've sent out. It's so weird that its capturing all recent magnets but won't show them.

ghost commented 10 months ago

RARBG was sooo good. But love DHT search engines. I can only hope they are busy and will return but not even acknowledging the emails I've sent out. It's so weird that its capturing all recent magnets but won't show them.

Yes, they were an exception to the rule.

Going to look back into running the crawler locally this weekend i think.

DarthAdriano commented 10 months ago

RARBG was sooo good. But love DHT search engines. I can only hope they are busy and will return but not even acknowledging the emails I've sent out. It's so weird that its capturing all recent magnets but won't show them.

Yes, they were an exception to the rule.

Going to look back into running the crawler locally this weekend i think.

local crawler? like a DHT search but or own, we can do that?!

ghost commented 10 months ago

local crawler? like a DHT search but or own, we can do that?! Ummm that is the entire point of this repository... ( unless you were being sarcastic, if so i missed the joke )

DarthAdriano commented 10 months ago

local crawler? like a DHT search but or own, we can do that?! Ummm that is the entire point of this repository... ( unless you were being sarcastic, if so i missed the joke )

lol nvm

DarthAdriano commented 10 months ago

BTdig working now!