NexvisionLab / Darkweb-search-engine

Dark Web & Deep Web Search Engine. Data Crawler and indexer for Darkweb , OSINT Tools for the Dark Web
GNU Affero General Public License v3.0
66 stars 14 forks source link

Problem in initializing the indexes on Elasticsearch #1

Closed shivaabed1994 closed 2 years ago

shivaabed1994 commented 2 years ago

Hi, I have worked on launching this search engine, but I have some questions. First: When the last command (docker exec nexvisions-torscraper-onion-crawler /opt/torscraper/scripts/elasticsearch_migrate.sh) run I got this error: Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 169, in _new_conn conn = connection.create_connection( File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 73, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/usr/local/lib/python3.8/socket.py", line 918, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/elasticsearch/connection/http_urllib3.py", line 241, in perform_request response = self.pool.urlopen( File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 755, in urlopen retries = retries.increment( File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 506, in increment raise six.reraise(type(error), error, _stacktrace) File "/usr/local/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, in reraise raise value File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 699, in urlopen httplib_response = self._make_request( File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 394, in _make_request conn.request(method, url, **httplib_request_kw) File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 234, in request super(HTTPConnection, self).request(method, url, body=body, headers=headers) File "/usr/local/lib/python3.8/http/client.py", line 1256, in request self._send_request(method, url, body, headers, encode_chunked) File "/usr/local/lib/python3.8/http/client.py", line 1302, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/usr/local/lib/python3.8/http/client.py", line 1251, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/usr/local/lib/python3.8/http/client.py", line 1011, in _send_output self.send(msg) File "/usr/local/lib/python3.8/http/client.py", line 951, in send self.connect() File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 200, in connect conn = self._new_conn() File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 181, in _new_conn raise NewConnectionError( urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f5dc663f4c0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/torscraper/scripts/elasticsearch_migrate.py", line 3, in tor_elasticsearch.migrate() File "/opt/torscraper/lib/tor_elasticsearch.py", line 463, in migrate entity_store.delete(ignore=404) File "/usr/local/lib/python3.8/site-packages/elasticsearch_dsl/index.py", line 376, in delete return self._get_connection(using).indices.delete(index=self._name, *kwargs) File "/usr/local/lib/python3.8/site-packages/elasticsearch/client/utils.py", line 84, in _wrapped return func(args, params=params, **kwargs) File "/usr/local/lib/python3.8/site-packages/elasticsearch/client/indices.py", line 259, in delete return self.transport.perform_request( File "/usr/local/lib/python3.8/site-packages/elasticsearch/transport.py", line 351, in perform_request status, headers_response, data = connection.perform_request( File "/usr/local/lib/python3.8/site-packages/elasticsearch/connection/http_urllib3.py", line 254, in perform_request raise ConnectionError("N/A", str(e), e) elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7f5dc663f4c0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7f5dc663f4c0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution)

What can I do to resolve this error?

NexvisionLab commented 2 years ago

Hi, @shivaabed1994, it looks like the elasticsearch container is not up in your environment. Can you check its status and share the log of elasticsearch instances?

shivaabed1994 commented 2 years ago

Hi @NexvisionLab Yes I checked those elasticsearch logs and fix the problem. The problem occurred because of volume commands in Docker file.