MHMDhub / enterprise-log-search-and-archive

Automatically exported from code.google.com/p/enterprise-log-search-and-archive
0 stars 0 forks source link

Override $Max_limit and $Unbatched_results_limit in elsa_web.conf #142

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago
We run large reports through the API and in order to get results I need to 
override $Max_limit in API.pm and $Unbatched_results_limit in Results.pm. Every 
time I update Elsa I needed to remake these changes, it would be nice if this 
could be configurable in elsa_web.conf.

Also, in Utils.pm, the http_post call in _peer_query has a default timeout of 5 
minutes. For the long running queries using the API the timeout hits before 
results are returned. I've modified the line to the following, but this would 
also be nice to have a configuration option in elsa_web.conf.

$q->peer_requests->{$peer} = http_post $url, $request_body, headers => 
$headers, timeout => 1800, sub { ...

Original issue reported on code.google.com by and...@wolftrail.com on 17 May 2013 at 3:39

GoogleCodeExporter commented 8 years ago
Is there a reason you can't use batch queries for this?  You set analytics=1 or 
limit=0 to have the query batch.  It will go into the background and email you 
when the result is complete.  The results will be in bulk format for easy 
downloading.

Original comment by mchol...@gmail.com on 17 May 2013 at 9:45

GoogleCodeExporter commented 8 years ago
My script takes the output of the large queries and does further postprocessing 
on it. That's why I need it to all be 100% automated. It works fine so long as 
Elsa is updated to allow larger result sets and timeouts. 

For users accessing the site via the web interface, I prefer it to go to batch 
and email them the results.

Original comment by and...@wolftrail.com on 17 May 2013 at 10:04

GoogleCodeExporter commented 8 years ago
Ah, ok, then try both analytics:1 and nobatch:1 so that the query runs in the 
foreground for your script.  You might wish to use the contrib/bulk_query.pl 
script as a template for the apparatus around the query itself.

Original comment by mchol...@gmail.com on 17 May 2013 at 10:10