search

iudicium / pryingdeep

Prying Deep - An OSINT tool to collect intelligence on the dark web.
GNU General Public License v3.0
330 stars 27 forks source link

docker version manual? #13

Open devcomputer opened 3 months ago

devcomputer commented 3 months ago

i run your docker inside an lxc and when I run docker-compose up i see a never ending list of crawls to iana and rfc-editor and so on. no idea how long this is going to take but its quite a while now and I have no idea if this is part of the first start or if need to pass any arguments on start. sth. like "go edit pryingdeep.yaml first"?

iudicium commented 3 months ago

It's not part of the first start. The crawler finds them as it goes. Could you share how some of the infinite links look like for e.g iana or rfc-editor? A quick fix right now would be to add these infinite websites to dissalowed-domains in pryingdeep.yaml

fabstest commented 2 months ago

I think he is referring to this one. I encounter the same issue

image

iudicium commented 2 months ago

I think he is referring to this one. I encounter the same issue

image

Oh that happens because the default url in the config Is https://example.com Remove it and it won't go to iana anymore or pass new urls with this arg

fabstest commented 2 months ago

After removing it I'll just run docker compose up then it'll be fine? Or I still need to rebuild?

devcomputer commented 4 weeks ago

i tried some debian based machines but they all also fail using the docker-compose.yml

image

when I uncomment the healthcheck the containers start and when editing the config to another URL it all just crashed again...but with new errors. ¯_(ツ)_/¯

iudicium commented 3 weeks ago

After removing it I'll just run docker compose up then it'll be fine? Or I still need to rebuild?

Yeah, you need to rebuild. Make sure you give a starting link if you're using the crawl command

iudicium commented 3 weeks ago

i tried some debian based machines but they all also fail using the docker-compose.yml

image

when I uncomment the healthcheck the containers start and when editing the config to another URL it all just crashed again...but with new errors. ¯(ツ)/¯

Maybe there's some issue with the healthcheck that I'm doing here. Make sure you re-build with the new urls though. I'll try to add some explanation to Docker section within a few days and check the healthcheck. For now, can you paste the traceback of the new errors here?