search

vlall / Darksearch

:mag::shipit: Search engine for hidden material. Scraping dark web onions, irc logs, deep web etc...
http://darksearch.com
466 stars 85 forks source link

Can't search for specific keywords #8

Open anishmi123 opened 7 years ago

anishmi123 commented 7 years ago

Hi Vlad,

Wishing you a very Happy New Year and hope the new year brings DarkSearch more success. I was just trying out your updated platform and I saw that while searches for generic keywords such as passwords, accounts worked fine, search for some keywords specific to a company name yielded no results. However, when searched for the same keyword on the sites like onion.city / onion.link yielded results.

Also, searches for phrases like two words matched only the first word in the phrase.

Regards, Anish

ShaliDaliHaliDa commented 7 years ago

HI Ansih, did you setup elastic search? No searches work for me how are you populating your DB?

anishmi123 commented 7 years ago

I was trying to use their web portal.. Regards, Anish Mitra

On 01-Apr-2017 8:01 PM, "ShaliDaliHaliDa" notifications@github.com wrote:

HI Ansih, did you setup elastic search? No searches work for me how are you populating your DB?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/vlall/darksearch/issues/8#issuecomment-290932864, or mute the thread https://github.com/notifications/unsubscribe-auth/AR3lgDC05ZZfFye-9CXCdNx5-1Tc0Ykjks5rroLegaJpZM4Lag3l .

ShaliDaliHaliDa commented 7 years ago

Thanks. I installed scraps, elasticsearch in an effort to learn more but whenever I search it get an error saying:

elasticsearch.exceptions.NotFoundError

NotFoundError: TransportError(404, u'index_not_found_exception', u'no such index')

I think that's because although I am running elastic search -- it's empty! Wondering how do I get it to populate?

@vlall ?

glerinsonum commented 7 years ago

@ShaliDaliHaliDa @vlall

Can you learn me, how I can integrating for executing, these programming scripts, into Maltego Interface. Thank you cordially.

vlall commented 6 years ago

@glerinsonum I just googled Maltego, and it looks very interesting for network analysis. If this is still something you're interested in pursuing, let me know. What other tools are you interesting in using for analysis? I would rather compare a list of these tools and work on porting it to the quickest one with the largest open source community first.

vlall commented 6 years ago

@ShaliDaliHaliDa There is a separate /Crawler library I built in a private Gitlab repo that I'm looking to merge and integrate with testing. The crawler was self-made so it's not optimized and a little buggy. This was originally going to be replaced by an open source more advanced crawler like Scrapy or Nutch. The crawler grabs the necessary links, then populates them into a Postgres database for long-term storage... Then into Elasticsearch for near-term retrieval.