search

TravisFSmith / SweetSecurity

Network Security Monitoring on Raspberry Pi type devices
Apache License 2.0
777 stars 190 forks source link

No matching indices found: No indices match pattern "logstash-*" #22

Open xneo1 opened 7 years ago

xneo1 commented 7 years ago

After installing it in Linux Mint 18 (VM) I access Kibana and it shows "No matching indices found: No indices match pattern "logstash-*"".

The default index in Advanced Settings is: logstash-*

Nothing is discovered. Also I cannot access the Sweet Security WebApp .(As it's a testing machine I used the same pwd for webapp and elastic )

Do you know how I can fix it? On the vm I can ping successfully the other network devices.

xneo1 commented 7 years ago

I launched the script again, changed the password (removed a number but kept the same pwd for both elastic and apache/kibana) but I still have the same problem. it shows "Internal Server Error:

pdobrien3 commented 7 years ago

I have this same issue. First with a client/server install on two raspberry pis and second with a ubuntu 16.04 full install on a laptop.

pdobrien3 commented 7 years ago

This is the only error I got on the second install:

WARNING: can not set Session#timeout=(0) no session context.

TravisFSmith commented 7 years ago

@xneo1 the matching indices message means that Logstash has not sent any data up to the server. I would ensure that logstash is running and configured correctly first. Run 'sudo service logstash status' on the client to make sure it is running. If not, run 'sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf -t' to ensure that the configuration file did not get corrupted.

As far as the internal server error, I would need to see the error message from /var/log/apache2/error.log to be able to troubleshoot that error.

diegodonos commented 7 years ago

Hi, I have the same problem with logstash, when I run the command that you put in the other comment, the result this is:

_ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console. WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at path //usr/share/logstash/config/log4j2.properties. Using default config which logs to console 12:52:23.192 [LogStash::Runner] ERROR logstash.plugins.registry - Problems loading a plugin with {:type=>"output", :name=>"email", :path=>"logstash/outputs/email", :error_message=>"NameError", :error_class=>NameError, :error_backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:226:in namespace_lookup'", "/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:162:inlegacy_lookup'", "/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:138:in lookup'", "/usr/share/logstash/logstash-core/lib/logstash/plugins/registry.rb:180:inlookup_pipelineplugin'", "/usr/share/logstash/logstash-core/lib/logstash/plugin.rb:140:in lookup'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:100:inplugin'", "(eval):1730:in initialize'", "org/jruby/RubyKernel.java:1079:ineval'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:72:in initialize'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:299:inexecute'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:209:inrun'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/share/logstash/lib/bootstrap/environment.rb:71:in(root)'"]} 12:52:23.196 [LogStash::Runner] FATAL logstash.runner - The given configuration is invalid. Reason: Couldn't find any output plugin named 'email'. Are you sure this is correct? Trying to load the email output plugin resulted in this error: Problems loading the requested plugin named email of type output. Error: NameError NameError

And the Apache error this is:

_[Wed Sep 06 11:11:14.726488 2017] [wsgi:error] [pid 1484:tid 139638001870592] WARNING:elasticsearch:GET http://localhost:9200/sweet_security/devices/_search?size=1000 [status:N/A request:10.010s] [Wed Sep 06 11:11:14.847946 2017] [wsgi:error] [pid 1484:tid 139638001870592] Traceback (most recent call last): [Wed Sep 06 11:11:14.847967 2017] [wsgi:error] [pid 1484:tid 139638001870592] File "/usr/local/lib/python2.7/dist-packages/elasticsearch/connection/http_urllib3.py", line 114, in perform_request [Wed Sep 06 11:11:14.847971 2017] [wsgi:error] [pid 1484:tid 139638001870592] response = self.pool.urlopen(method, url, body, retries=False, headers=self.headers, **kw) [Wed Sep 06 11:11:14.847974 2017] [wsgi:error] [pid 1484:tid 139638001870592] File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 639, in urlopen [Wed Sep 06 11:11:14.847976 2017] [wsgi:error] [pid 1484:tid 139638001870592] _stacktrace=sys.exc_info()[2]) [Wed Sep 06 11:11:14.847979 2017] [wsgi:error] [pid 1484:tid 139638001870592] File "/usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py", line 333, in increment [Wed Sep 06 11:11:14.847981 2017] [wsgi:error] [pid 1484:tid 139638001870592] raise six.reraise(type(error), error, _stacktrace) [Wed Sep 06 11:11:14.847983 2017] [wsgi:error] [pid 1484:tid 139638001870592] File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 601, in urlopen [Wed Sep 06 11:11:14.847986 2017] [wsgi:error] [pid 1484:tid 139638001870592] chunked=chunked) [Wed Sep 06 11:11:14.847988 2017] [wsgi:error] [pid 1484:tid 139638001870592] File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 389, in _make_request [Wed Sep 06 11:11:14.847991 2017] [wsgi:error] [pid 1484:tid 139638001870592] self._raise_timeout(err=e, url=url, timeout_value=read_timeout) [Wed Sep 06 11:11:14.847993 2017] [wsgi:error] [pid 1484:tid 139638001870592] File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 309, in _raise_timeout [Wed Sep 06 11:11:14.847996 2017] [wsgi:error] [pid 1484:tid 139638001870592] raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeoutvalue) [Wed Sep 06 11:11:14.847998 2017] [wsgi:error] [pid 1484:tid 139638001870592] ReadTimeoutError: HTTPConnectionPool(host='localhost', port=9200): Read timed out. (read timeout=10)

Many thanks!

everdult commented 6 years ago

having the same problem... tried on several machine to do a full or a splitted config but always the logstash pattern error. Logstash is running

buckshome commented 6 years ago

If there are any errors at all (even if's running) logstash won't parse any messages. I had mentioned this in another thread but the current version of logstash no longer comes with email functionality built in. You probably configured logstash to send you emails during installation. You'll have to install the email plugin bin/logstash-plugin install logstash-output-email

Agromahdi123 commented 6 years ago

Ok, i really dont know much about coding, but i found an easy fix for this. The installer has a python script to import indices, so what i did was run curl -XDELETE http://localhost:9200/.kibana to delete the indices created by ES that somehow break kibana, and then rerun the install script, and it will reimport the indices and the dashboards, and they will work.

mlinton commented 6 years ago

For what ever reason - the installation script failed for me too and logstash even though its installed does not have any of the configuration files in the right spots (e.g. empty /etc/logstash/conf.d/). Looking at the python logstash install script it looks like there is a bunch of stuff being done. I would be nice to be able to just rerun the logstash install to try to get it all working. Any ideas?

thatsatechnique commented 6 years ago

sudo /usr/share/logstash/bin/logstash-plugin install logstash-output-email

That's the command to install the email plugin if you are having issues getting your indices to show up from client --> Server (missing any logstash entries in Kibana).

InfoSecured commented 6 years ago

I had this problem too. Mkiukaji's response worked for me. I had to restart both devices though before it started working.

jouellnyc commented 5 years ago

@xneo1 the matching indices message means that Logstash has not sent any data up to the server. I would ensure that logstash is running and configured correctly first. Run 'sudo service logstash status' on the client to make sure it is running. If not, run 'sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf -t' to ensure that the configuration file did not get corrupted.

As far as the internal server error, I would need to see the error message from /var/log/apache2/error.log to be able to troubleshoot that error.

Same issue here - Though I get:


root@raspberrypi:/opt/nsm/bro/logs/current#  /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf -t
io/console on JRuby shells out to stty for most operations
Bundler::GemNotFound: Could not find gem 'logstash-filter-translate (>= 0) java' in any of the gem sources listed in your Gemfile or installed on this machine.
  verify_gemfile_dependencies_are_found! at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/resolver.rb:328
                                    each at org/jruby/RubyArray.java:1613
  verify_gemfile_dependencies_are_found! at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/resolver.rb:307
                                   start at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/resolver.rb:199
                                 resolve at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/resolver.rb:182
                                 resolve at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/definition.rb:192
                                   specs at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/definition.rb:132
                               specs_for at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/definition.rb:177
                         requested_specs at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/definition.rb:166
                         requested_specs at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/environment.rb:18
                                   setup at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler/runtime.rb:13
                                   setup at /usr/share/logstash/vendor/bundle/jruby/1.9/gems/bundler-1.9.10/lib/bundler.rb:122
                                  setup! at /usr/share/logstash/lib/bootstrap/bundler.rb:67
                                  (root) at /usr/share/logstash/lib/bootstrap/environment.rb:67