Closed certrik closed 1 year ago
Another sidenote: I am also using the env variables http_proxy, https_proxy and no_proxy, where no_proxy
contains localhost,127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,172.17.0.0/16
In order to have working crawlers with the script from ail-framework I had to change splash_url = http://127.0.0.1
to splash_url = http://localhost
in ail-framework/configs/core.cfg
under [Crawler]
Hi @Terrtia , is the way the crawlers are started somehow different to the way in the script form the ail-framework? I tried the spalsh-manager and everything seems to start right but the crawlers seem to not to be able to connect via the TOR proxy. This is not the case when using the script from the ail-framework. Sidenote: I am using TOR behind a http(s) proxy which is configured in
/etc/tor/torrc
.