search

ail-project / ail-splash-manager

Deprecated: AIL crawler has been upgraded to https://github.com/ail-project/lacus
GNU General Public License v3.0
4 stars 2 forks source link

Question: Initialisation in comparison with script from ail-framework #1

Closed certrik closed 1 year ago

certrik commented 4 years ago

Hi @Terrtia , is the way the crawlers are started somehow different to the way in the script form the ail-framework? I tried the spalsh-manager and everything seems to start right but the crawlers seem to not to be able to connect via the TOR proxy. This is not the case when using the script from the ail-framework. Sidenote: I am using TOR behind a http(s) proxy which is configured in /etc/tor/torrc .

certrik commented 4 years ago

Another sidenote: I am also using the env variables http_proxy, https_proxy and no_proxy, where no_proxy contains localhost,127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,172.17.0.0/16 In order to have working crawlers with the script from ail-framework I had to change splash_url = http://127.0.0.1 to splash_url = http://localhost in ail-framework/configs/core.cfg under [Crawler]

Terrtia commented 1 year ago

AIL v5.0 AIL crawler has been upgraded to Lacus.

Lacus operates independently of AIL and can run on the same server or a different one. AIL is just fetching the crawled content from lacus.