Cuckoo Elastic Search Installation Ubuntu

[seantree]

Hi

I am trying to install Elastic Search on ubuntu 14.04 but not getting success I have installed the latest version of Cuckoo sandbox successfully but not able to getting success installing elastic search 5.5. . It would be very helpful if someone guide me in how to configure cuckoo sandbox and elastic search 5.5 . Thanks & Regards Sean

[jbremer]

The elasticsearch aspect is still WIP / a beta feature. Anyone want to chime in here, @KillerInstinct @swackhamer @RicoVZ @razuz @SparkyNZL?

[SparkyNZL]

Hi Seantree,

So you have installed ES is the service active (service elasticsearch status)

if that is up great. You should beable to curl the api with the following

curl localhost:9200

the output should look something like the following

---

name	"MalwareLab"
cluster_name	"LabbyMcLablab"
cluster_uuid	"J0LtIXfvT0Kn5F3324hlrg"
version
number	"5.3.1"
build_hash	"5f9cf58"
build_date	"2017-04-17T15:52:53.846Z"
build_snapshot	false
lucene_version	"6.4.2"
tagline	"You Know, for Search"

---

Let me know if you get this far, if not we can go back to the installation and config of the elasticsearch.yml in the /etc/elasticsearch

P.S. ES needs sh*t loads of ram, like minimum of 24g if you dont have that avaliable then running it with Cuckoo is pointless as the Java will keep running out of memory and the service will just STOP :(

Cheers S

[seantree]

Hi @SparkyNZL ,

It would be great and helpful for me and others new users also if you guide me the installation process of Elastic Search(ES), I know its available on the internet and youtube also. I followed those guides and faces lots of problems while installation so that's why I am asking if you guide me in the installation of ES also, I know it will be hectic for you sorry for that, but it will be helpful for future users also. . I hope you will understand this part & guide me.. . Thanks & Regards Seantree

[SparkyNZL]

You will need to download the .deb from elasticsearch, you will need to install java as well the install in the deb package with

sudo dpkg -i elasticsearch.deb Install Java

You will need to edit the elasticsearch.yml file located in /etc/elasticsearch

Hope this helps.

[seantree]

Hi @SparkyNZL,

I have already installed Java & Elastic Search now the thing is what to configure in elasticsearch.yml file.

When I try to execute the following command it shows the following error: curl localhost:9200

curl: (7) Failed to connect to localhost port 9200: Connection refused. . It would be great if you help me in configuration part. . Thanks & Regards Seantree

[seantree]

Hi @SparkyNZL

By adding the following line in elasticsearch.yml network.host: 0.0.0.0, the curl command executed & it shows the following result: curl localhost:9200 { "name" : "ge0bOdk", "cluster_name" : "elasticsearch", "cluster_uuid" : "CHa3DmjrS3Wyj_asEHDryg", "version" : { "number" : "5.5.0", "build_hash" : "260387d", "build_date" : "2017-06-30T23:16:05.735Z", "build_snapshot" : false, "lucene_version" : "6.6.0" }, "tagline" : "You Know, for Search" } Now tell me what to do further, to interact with the cuckoo and what are the changes that I have to do in elasticsearch.yml if any needed. . Thanks & Regards Seantree

[SparkyNZL]

Great work,

Now you need to edit the configs for cuckoo, so you can use your fav editor

you need to edit reporting.conf, scroll down to elasticsearch and change "enabled = no" to "enabled = yes" , then change the entry doe the host, hosts = 127.0.0.1:9200

start up cuckoo and you should be all done, if you find that the elasticsearch service keeps stopping you will need to edit the jvm.options file, and increase the heapspace, i have mind set to 20g so change -XmsXXg to -Xms20g for both options.

Hope this helps

On Tue, Jul 18, 2017 at 7:16 PM, seantree notifications@github.com wrote:

Hi @SparkyNZL https://github.com/sparkynzl

By adding the following line in elasticsearch.yml network.host: 0.0.0.0, the curl command executed & it shows the following result: curl localhost:9200 { "name" : "ge0bOdk", "cluster_name" : "elasticsearch", "cluster_uuid" : "CHa3DmjrS3Wyj_asEHDryg", "version" : { "number" : "5.5.0", "build_hash" : "260387d", "build_date" : "2017-06-30T23:16:05.735Z", "build_snapshot" : false, "lucene_version" : "6.6.0" }, "tagline" : "You Know, for Search" } Now tell me what to do further, to interact with the cuckoo and what are the changes that I have to do in elasticsearch.yml if any needed. . Thanks & Regards Seantree

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cuckoosandbox/cuckoo/issues/1704#issuecomment-315978922, or mute the thread https://github.com/notifications/unsubscribe-auth/AQ_imKBoWc6oFAM0Qo8T89jGwbOELkiNks5sPFvFgaJpZM4OT5N7 .

[seantree]

Hi @SparkyNZL

I am getting this error:

||| | | ||| | | || || | | | | || | | | | | | | | | | | | | | ||| ||| ||| | | || |_|

Cuckoo Sandbox 2.0.3 www.cuckoosandbox.org Copyright (c) 2010-2017

2017-07-19 16:10:58,045 [cuckoo.common.config] ERROR: Type of config parameter processing:procmon:procmon not found! This may indicate that you've incorrectly filled out the Cuckoo configuration, please double check it. 2017-07-19 16:10:58,045 [cuckoo.common.config] ERROR: Config section processing:foobar not found! Checking for updates... You're good to go! 2017-07-19 16:11:00,996 [elasticsearch] WARNING: HEAD http://127.0.0.1:9200/_template/cuckoo_template [status:N/A request:0.000s] Traceback (most recent call last): File "/home/sean/venv/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 114, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, kw) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 649, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 356, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python2.7/httplib.py", line 1017, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1051, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1013, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 864, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 826, in send self.connect() File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) NewConnectionError: <urllib3.connection.HTTPConnection object at 0xb32705ec>: Failed to establish a new connection: [Errno 111] Connection refused 2017-07-19 16:11:00,998 [elasticsearch] WARNING: HEAD http://127.0.0.1:9200/_template/cuckoo_template [status:N/A request:0.000s] Traceback (most recent call last): File "/home/sean/venv/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 114, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, kw) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 649, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 356, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python2.7/httplib.py", line 1017, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1051, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1013, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 864, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 826, in send self.connect() File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) NewConnectionError: <urllib3.connection.HTTPConnection object at 0xb32701ec>: Failed to establish a new connection: [Errno 111] Connection refused 2017-07-19 16:11:00,998 [elasticsearch] WARNING: HEAD http://127.0.0.1:9200/_template/cuckoo_template [status:N/A request:0.000s] Traceback (most recent call last): File "/home/sean/venv/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 114, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, kw) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 649, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 356, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python2.7/httplib.py", line 1017, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1051, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1013, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 864, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 826, in send self.connect() File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) NewConnectionError: <urllib3.connection.HTTPConnection object at 0xb32705ec>: Failed to establish a new connection: [Errno 111] Connection refused 2017-07-19 16:11:00,999 [elasticsearch] WARNING: HEAD http://127.0.0.1:9200/_template/cuckoo_template [status:N/A request:0.000s] Traceback (most recent call last): File "/home/sean/venv/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 114, in perform_request response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, kw) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 649, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/util/retry.py", line 333, in increment raise six.reraise(type(error), error, _stacktrace) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connectionpool.py", line 356, in _make_request conn.request(method, url, httplib_request_kw) File "/usr/lib/python2.7/httplib.py", line 1017, in request self._send_request(method, url, body, headers) File "/usr/lib/python2.7/httplib.py", line 1051, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1013, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 864, in _send_output self.send(msg) File "/usr/lib/python2.7/httplib.py", line 826, in send self.connect() File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connection.py", line 166, in connect conn = self._new_conn() File "/home/sean/venv/local/lib/python2.7/site-packages/urllib3/connection.py", line 150, in _new_conn self, "Failed to establish a new connection: %s" % e) NewConnectionError: <urllib3.connection.HTTPConnection object at 0xb32701ec>: Failed to establish a new connection: [Errno 111] Connection refused Oops! Cuckoo failed in an unhandled exception! Sometimes bugs are already fixed in the development release, it is therefore recommended to retry with the latest development release available https://github.com/cuckoosandbox/cuckoo If the error persists please open a new issue at https://github.com/cuckoosandbox/cuckoo/issues

=== Exception details === Cuckoo version: 2.0.3 OS version: posix OS release: Ubuntu 14.04 trusty Python version: 2.7.6 Python implementation: CPython Machine arch: i686

Traceback (most recent call last): File "/home/sean/venv/local/lib/python2.7/site-packages/cuckoo/main.py", line 214, in main cuckoo_init(level, ctx) File "/home/sean/venv/local/lib/python2.7/site-packages/cuckoo/main.py", line 123, in cuckoo_init init_modules() File "/home/sean/venv/local/lib/python2.7/site-packages/cuckoo/core/startup.py", line 190, in init_modules module.init_once() File "/home/sean/venv/local/lib/python2.7/site-packages/cuckoo/reporting/elasticsearch.py", line 50, in init_once if not elastic.client.indices.exists_template(cls.template_name): File "/home/sean/venv/local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped return func(*args, params=params, **kwargs) File "/home/sean/venv/local/lib/python2.7/site-packages/elasticsearch/client/indices.py", line 475, in exists_template name), params=params) File "/home/sean/venv/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout) File "/home/sean/venv/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 123, in perform_request raise ConnectionError('N/A', str(e), e) ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0xb32701ec>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0xb32701ec>: Failed to establish a new connection: [Errno 111] Connection refused)

---

and kindly tell me where is the jvm.options file & how to change it. . Thanks & Regards Seantree

[SparkyNZL]

hte jvm.options file is in the /etc/elasticsearch directory.

can you do a curl 127.0.0.1:9200

if you dont get any response can you change in your elasticsearch.yml file the network.host: 0.0.0.0 to 127.0.0.1

then try again, you might want to also upgrade your cuckoo to the latest

sudo pip install cuckoo==2.0.4a5

Cheers

[seantree]

Hi @SparkyNZL,

Same above error is generating and now elastic search also giving the error of connections refused . Thanks & Regards Seantree

[SparkyNZL]

change the network,host to your primary IP address, me the changes in your reporting.conf to reflect this. You will need to stop the Elasticsearch service and restart by typeing sudo service elasticsearch restart

then curl ipaddress:9200

if you get response from ES great if not check to make sure that you arent running UFW as this will block the port.

to find out your primary IP do a ifconfig and it will be listed in there.

[seantree]

Hi, Little bit confusion here I am little bit weak in networking so just asking I didn't get this part "change the network host to your primary ip address". Would you please tell me in detail, if you don't mind

[SparkyNZL]

ok so if you open a terminal and type in ifconfig the output will list all your network cards and IP addresses in there will have the IP address for Eth0 or ens XXXXX (x = numbers) if you take note of this IP address and put that in your es config this will cause es to bind to the ip address

then you should be able to curl ipaddress:9200

once you can do this, change the config of Cuckoo to point at that ip address and you should be underway.

[SparkyNZL]

How did you get on @seantree