thalesgroup-cert / Watcher

Watcher - Open Source Cybersecurity Threat Hunting Platform. Developed with Django & React JS.
https://thalesgroup-cert.github.io/Watcher
GNU Affero General Public License v3.0
850 stars 122 forks source link

Scrape error Watcher #34

Closed m1chas closed 3 years ago

m1chas commented 3 years ago

I am getting this output when watcher is scraping for dataleaks.

watcher | 2021-04-19 22:50:08.063406 - HTTPSConnectionPool(host='10.10.10.3', port=8080): Max retries exceeded with url: /search?q=%22xxxxx%22&engines=gitlab%2Cgithub%2Cbitbucket%2Capkmirror%2Cgentoo%2Cnpm%2Cstackoverflow%2Choogle&format=json (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:1125)')))

Felix83000 commented 3 years ago

Hi,

Do you get this output for all requests? Or just one or two?

Regards,

m1chas commented 3 years ago

Hi,

I get the output for all requests (except pastebin which gives the pro warning in addition).

Thank you,

Felix83000 commented 3 years ago

I will check why this happened and come back to you this week.

Regards,

m1chas commented 3 years ago

thank you very much. (love the app btw)

Felix83000 commented 3 years ago

It is due to a major change with the new Searx version... Please use the new docker-compose.yml. (https://github.com/thalesgroup-cert/Watcher/blob/master/docker-compose.yml)

Felix83000 commented 3 years ago

Please tell me if it worked.

Regards,

m1chas commented 3 years ago

nope, exactly the same error. sorry.

did a complete reinstall.

Felix83000 commented 3 years ago

Please follow these steps:

docker-compose down
docker rmi searx/searx

In the docker-compose.yml file change image: searx/searx:latest by image: searx/searx:0.18.0-341-ae0b621e

Be sure that your settings.yml is exactly the same than this one: https://www.github.com/thalesgroup-cert/Watcher/tree/master/Searx%2Fsearx%2Fsettings.yml

docker-compose up

It worked on my side, I hope it will work on your side. Keep me posted.

m1chas commented 3 years ago

hey Felix, tks! It works now.

Felix83000 commented 3 years ago

Thank YOU for reporting this issue! 👍