Can't see any new queries

[GoogleCodeExporter]

What is the expected output? What do you see instead?
My elsa node instance has been collecting bro and Snare logs for quite some 
time, but AFAIK no new events have been added to the event database since 
December 31st. I'd love some help figuring out why new events are disappearing.

What version of the product are you using? On what operating system?
NODE: Archlinux (latest release, latest SVN)
WEB: CentOS (6.2, latest SVN via update script)

Please provide any additional information below.
I will provide any configuration files you need.

Original issue reported on code.google.com by i...@pingas.org on 9 Jan 2013 at 7:26

[GoogleCodeExporter]

Ok, basics to check:
Is disk full?
Are services running (mysql, syslog-ng, searchd)?
What is in /data/elsa/log/node.log?

Original comment by mchol...@gmail.com on 9 Jan 2013 at 7:39

[GoogleCodeExporter]

The data dir's disk is an iSCSI mount to a 1.5 TB volume. df -h says it has 
1.4T free.

MySql.service is running on the node instance, as are syslog-ng and searchd. 
(At least, systemd claims they are. Their respective ports are also open and 
accepting data).

My datadir is /srv/, but the node.log file seems to be 18 GB big... Would it be 
better to just give you the last 10,000 lines or so? I've attached a file with 
the last 10,000 lines.

Original comment by i...@pingas.org on 12 Jan 2013 at 7:23

Attachments:

• stdin.txt

[GoogleCodeExporter]

Ok, it looks like some process is going haywire and continually reindexing.  
Shut down syslog-ng and make sure there are no Perl processes running.  It's 
also odd that Sphinx is using /etc/sphinx.conf.  Did you move the location from 
/usr/local/etc/sphinx.conf, which is the ELSA default?

After syslog-ng is shut down and no Perl processes are running, run:

cd /usr/local/elsa/node && echo "" | perl elsa.pl -o

That should complete one full ELSA batch cycle and return with no errors or 
warnings.

Original comment by mchol...@gmail.com on 12 Jan 2013 at 9:36

[GoogleCodeExporter]

I'm pretty sure I copied over the /usr/local/etc/sphinx.conf file to 
/etc/sphinx.conf (and modified configs accordingly), since I wanted to have 
sphinx's conf in the upstream location. I've attached it. If it looks totally 
wrong, I'm fine with changing it to whatever it needs to be.

Syslog-ng.service is now shutdown. By the way, my ELSA dir is /usr/share/elsa. 
Sorry to be so non-standard here!
I've run the command (with the cd path changed), and the output is below:
--------------------------------------------------------------------------
root@loki ~ # cd /usr/share/elsa/node && echo "" | perl elsa.pl -0
Cannot read config file: /etc/elsa_node.conf at 
/usr/share/perl5/site_perl/Config/JSON.pm line 49, <DATA> line 429.
    Config::JSON::__ANON__('Config::JSON=HASH(0xa4c78f0)', '/etc/elsa_node.conf') called at constructor Config::JSON::new (defined at /usr/share/perl5/site_perl/Config/JSON.pm line 668) line 41
    Config::JSON::new('Config::JSON', '/etc/elsa_node.conf') called at elsa.pl line 52
--------------------------------------------------------------------------
I realized that my elsa configs are in /etc/elsa/ and not just /etc/, so I 
linked /etc/elsa/elsa_node.conf to /etc/elsa_node.conf and re-ran that command. 
It took much longer to run and the output is below:
--------------------------------------------------------------------------
root@loki elsa/node # cd /usr/share/elsa/node && echo "" | perl elsa.pl -o
Validating directory...
Use of uninitialized value $db_size in concatenation (.) or string at 
/usr/share/elsa/node/Indexer.pm line 156.
Use of uninitialized value $db_size in addition (+) at 
/usr/share/elsa/node/Indexer.pm line 175.
Use of uninitialized value $db_size in concatenation (.) or string at 
/usr/share/elsa/node/Indexer.pm line 191.
Use of uninitialized value $db_size in addition (+) at 
/usr/share/elsa/node/Indexer.pm line 211.
DBD::mysql::st execute failed: Table 'syslog.host_stats' doesn't exist at 
/usr/share/elsa/node/Indexer.pm line 253.
DBD::mysql::st execute failed: Table 'syslog.host_stats' doesn't exist at 
/usr/share/elsa/node/Indexer.pm line 253.
--------------------------------------------------------------------------
I just logged into the web frontend, but the tooltip still says the latest 
Index and Archive entries are from 2012-12-31. What's the next step? Should I 
just re-install elsa completely?

Original comment by i...@pingas.org on 14 Jan 2013 at 4:06

Attachments:

• etcsphinx.conf.txt

[GoogleCodeExporter]

Just realized -- that "-0" was actually a "-o" when I ran it. Not sure how it 
was changed in my comment. Sorry about that.

Original comment by i...@pingas.org on 14 Jan 2013 at 4:58

[GoogleCodeExporter]

Ok, good, we see the root of the problem "DBD::mysql..."  That error indicates 
that you haven't run the update, so you need to run "sh 
/usr/share/elsa/install.sh node update" to get that table installed so the rest 
of the code works.  No need to re-install.

Original comment by mchol...@gmail.com on 15 Jan 2013 at 1:16

[GoogleCodeExporter]

Closing due to inactivity.  Please comment to reopen.

Original comment by mchol...@gmail.com on 8 Apr 2013 at 2:05

• Changed state: Done

[GoogleCodeExporter]

Commenting just to say that it was eventually solved with an upgrade and a 
re-roll of the indexes. However, I'm having different problems now. I'm 
reopening a ticket to explain them.

Original comment by i...@pingas.org on 13 May 2013 at 5:48