search

SafeAF / enterprise-log-search-and-archive

Automatically exported from code.google.com/p/enterprise-log-search-and-archive
0 stars 0 forks source link

Valid Query Returns No Results #50

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
What steps will reproduce the problem?

This works: host:172.16.0.1 +class:url limit:1000 

This returns 0 results: 172.16.0.1 +class:url limit:10000

In reality, there are 1,460,234 records for this time period.

What is the expected output? What do you see instead?

I would expect to see 10,000 results or to have the query batched.

What version of the product are you using? On what operating system?

Latest SVN as of a couple of days ago, OEL 6.2

Original issue reported on code.google.com by lib...@gmail.com on 25 Jul 2012 at 2:42

GoogleCodeExporter commented 8 years ago 
Unfortunately, host is special because it is actually stored as an integer, not 
a string, and doesn't show up in the message.  When you search for the IP 
listed without the "host:" prefix, ELSA has to interpret it as a literal 
string, which won't match the integer representation in the host field.  I 
attempted a possible fix which would be to always search for both the IP and 
its integer representation, but that introduced other inconsistencies and 
unexpected search results.  I may look at this again, but right now I don't see 
a good way of fixing this.

Original comment by mchol...@gmail.com on 25 Jul 2012 at 5:24

GoogleCodeExporter commented 8 years ago 
Sorry, that was actually a copy and paste error. Host *is* in the second query, 
just like the first. The only difference is that I add a zero to the limit 
argument and then I get zero results.

Original comment by lib...@gmail.com on 25 Jul 2012 at 7:15

GoogleCodeExporter commented 8 years ago 
Ah, that's a very different problem!  So you see now debug messages or 
otherwise in the log to indicate that it tried to batch?

Original comment by mchol...@gmail.com on 25 Jul 2012 at 7:46

GoogleCodeExporter commented 8 years ago 
Here are the last few lines from the query that failed:

* TRACE [2012/07/25 15:06:15] /usr/local/elsa/web/lib/API.pm (2121) 
API::_unlimited_sphinx_query 1338 [undef]
total: 9967, overall_limit: 10000
* TRACE [2012/07/25 15:06:18] /usr/local/elsa/web/lib/API.pm (2143) 
API::_unlimited_sphinx_query 1338 [undef]
query got 1000 of 51822 results
* DEBUG [2012/07/25 15:06:18] /usr/local/elsa/web/lib/API.pm (2171) 
API::_unlimited_sphinx_query 1338 [undef]
found latest time: 1343155405 Tue Jul 24 13:43:25 2012
* DEBUG [2012/07/25 15:06:18] /usr/local/elsa/web/lib/API.pm (2186) 
API::_unlimited_sphinx_query 1338 [undef]
received: 10000 of 757126 with overall limit 10000
* DEBUG [2012/07/25 15:06:18] /usr/local/elsa/web/lib/API.pm (2193) 
API::_unlimited_sphinx_query 1338 [undef]
completed unlimited query in 39.5081880092621 with 10000 rows
* INFO [2012/07/25 15:06:19] /usr/local/elsa/web/lib/API.pm (1606) API::query 
1338 [undef]
Query 173 returned 0 rows

Here are the last few from the query that succeeds:

* TRACE [2012/07/25 15:08:17] /usr/local/elsa/web/lib/API.pm (1921) 
API::__ANON__ 1341 [undef]
node 127.0.0.1 got db rows: 1000
* DEBUG [2012/07/25 15:08:17] /usr/local/elsa/web/lib/API.pm (2097) 
API::_sphinx_query 1341 [undef]
completed query in 4.19301795959473 with 1000 rows
* INFO [2012/07/25 15:08:17] /usr/local/elsa/web/lib/API.pm (1606) API::query 
1341 [undef]
Query 174 returned 1000 rows

Prior to these entries there is a really big line with a bunch of question 
marks and integers, separated by commas.

Original comment by lib...@gmail.com on 25 Jul 2012 at 8:09

GoogleCodeExporter commented 8 years ago 
Ok, that's good, it did finish:
completed unlimited query in 39.5081880092621 with 10000 rows
So, there should be a CSV file somewhere (it's supposed to email you with those 
results) created in the bulk_dir (/tmp by default).

Original comment by mchol...@gmail.com on 25 Jul 2012 at 8:50

GoogleCodeExporter commented 8 years ago 
I don't see any *.csv files in /tmp, but there are a lot of files with names 
that look like MD5 checksums. Are those what I am looking for?

Original comment by lib...@gmail.com on 25 Jul 2012 at 11:06

GoogleCodeExporter commented 8 years ago 
The MD5 files are session stores and are unrelated.  And I was mistaken, the 
files are .json, not .csv.  Are there any .json files?

Original comment by mchol...@gmail.com on 26 Jul 2012 at 1:33

GoogleCodeExporter commented 8 years ago 
Closing due to inactivity.

Original comment by mchol...@gmail.com on 29 Nov 2012 at 10:35