ESS queries not working

[craigm-didata]

I'm running on Ubuntu 14 using Splunk 7.0.1.

When I run indices-list, I get the following (using the Collective Intelligence Framework's Elastic instance):

_time	mappings	number_of_replicas	aliases	number_of_shards	creation_date	uuid	name
2018-01-20 13:38:39	tokens	1		5	1516475310127	85PhRO_QQVq-4Vaw_leSkA	cif.tokens
2018-01-20 13:38:39	observables	1		5	1516476414154	jrsH4eNBRE2vqfDhue_VeQ	cif.observables-2018.01

When I run the following command line query, I see results:

curl -XGET 'http://localhost:9200/cif.observables-2018.01/_search?pretty=1&otype=fqdn' | more

When I run | ess eaddr="http://localhost:9200" index="cif.observables-2018.01" query="otype:fqdn", I get:

External search command 'ess' returned error code 1.

I tried to enable INFO level logging in the logging.conf file, but I never see any kind of log file show up in /opt/splunk/var/log/splunk.

How do I go about troubleshooting this?

[brunotm]

Hi, Can you post the search.log from the splunk job inspector?

Thanks

[craigm-didata]

Here is one of the searches:

| ess eaddr="http://localhost:9200/cif.observables-2018.01" query="otype:fqdn"

Here is the search job properties:

canSummarize None createTime 2018-01-22T10:09:09.000-08:00 cursorTime 2038-01-18T19:14:07.000-08:00 custom { [-]http://localhost:8000/en-US/manager/search/job_inspector?sid=1516644548.22# dispatch.earliest_time: 0 dispatch.latest_time: dispatch.sample_ratio: 1 display.general.type: statistics display.page.search.mode: smart display.page.search.tab: statistics search: | ess eaddr="http://localhost:9200/cif.observables-2018.01" query="otype:fqdn" }

defaultSaveTTL 604800 defaultTTL 600 delegate None diskUsage 114688 dispatchState DONE doneProgress 1 dropCount None eai:acl { [-]http://localhost:8000/en-US/manager/search/job_inspector?sid=1516644548.22# app: search can_write: true modifiable: true owner: admin perms: { [+]http://localhost:8000/en-US/manager/search/job_inspector?sid=1516644548.22# } sharing: global ttl: 600 }

earliestTime 1969-12-31T16:00:00.000-08:00 eventAvailableCount None eventCount None eventFieldCount None eventIsStreaming true eventIsTruncated true eventSearch None eventSorting desc isBatchModeSearch None isDone true isEventsPreviewEnabled None isFailed None isFinalized None isPaused None isPreviewEnabled true isRealTimeSearch None isRemoteTimeline None isSaved None isSavedSearch None isTimeCursored None isZombie None keywords None label None modifiedTime 2018-01-22T10:09:23.032-08:00 normalizedSearch None numPreviews None optimizedSearch | ess eaddr="http://localhost:9200/cif.observables-2018.01" query="otype:fqdn" pid 4858 priority 5 provenance UI:Search remoteSearch None reportSearch ess eaddr="http://localhost:9200/cif.observables-2018.01" query="otype:fqdn" request { [-]http://localhost:8000/en-US/manager/search/job_inspector?sid=1516644548.22# adhoc_search_level: smart auto_cancel: 30 check_risky_command: false custom.dispatch.earliest_time: 0 custom.dispatch.latest_time: custom.dispatch.sample_ratio: 1 custom.display.general.type: statistics custom.display.page.search.mode: smart custom.display.page.search.tab: statistics custom.search: | ess eaddr="http://localhost:9200/cif.observables-2018.01" query="otype:fqdn" earliest_time: 0 indexedRealtime: latest_time: preview: 1 provenance: UI:Search rf: * sample_ratio: 1 search: | ess eaddr="http://localhost:9200/cif.observables-2018.01" query="otype:fqdn" status_buckets: 300 ui_dispatch_app: search }

resultCount None resultIsStreaming None resultPreviewCount None runDuration 1.915207337 runtime { [-]http://localhost:8000/en-US/manager/search/job_inspector?sid=1516644548.22# auto_cancel: 30 auto_pause: 0 }

sampleRatio 1 sampleSeed 0 scanCount None search | ess eaddr="http://localhost:9200/cif.observables-2018.01" query="otype:fqdn" searchCanBeEventType None searchProviders [ ]

searchTotalBucketsCount None searchTotalEliminatedBucketsCount None sid 1516644548.22 statusBuckets None ttl 600 Additional info search.log http://localhost:8000/en-US/api/search/jobs/1516644548.22/search.log?outputMode=raw

---

From: Bruno Moura [notifications@github.com] Sent: Sunday, January 21, 2018 12:46 AM To: brunotm/elasticsplunk Cc: Craig Merchant (Americas); Author Subject: Re: [brunotm/elasticsplunk] ESS queries not working (#10)

Hi, Can you post the search.log from the splunk job inspector?

Thanks

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<redir.aspx?REF=9GXIUhVoqS3v9e1WPTtoPjCalrQabQPd6GBVZnu8NEGhZ4HEwmHVCAFodHRwczovL3Byb3RlY3QtZXUubWltZWNhc3QuY29tL3MvUnNjLUMzbHpQaHBLbzVrbmNxcEZhZQ..>, or mute the thread<redir.aspx?REF=X-qB32caTEuggRxNmUK6K1fEYc_XmbkQLHvnEniqPrahZ4HEwmHVCAFodHRwczovL3Byb3RlY3QtZXUubWltZWNhc3QuY29tL3MvVmlKd0M0OEFQRkI1RXBWWHRCOURUZg..>.

itevomcid

[brunotm]

This is the job inspector output, in the same window you’ll find a link for the search.log. Please attach it here.

Thanks

[craigm-didata]

OK. And how do I increase the log verbosity?

From: Bruno Moura [mailto:notifications@github.com] Sent: Tuesday, January 23, 2018 12:33 PM To: brunotm/elasticsplunk elasticsplunk@noreply.github.com Cc: Craig Merchant (Americas) craig.merchant@dimensiondata.com; Author author@noreply.github.com Subject: Re: [brunotm/elasticsplunk] ESS queries not working (#10)

This is the job inspector output, in the same window you’ll find a link for the search.log. Please attach it here.

Thanks

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/brunotm/elasticsplunk/issues/10#issuecomment-359921186, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfkbhHLK91fJXcXZVZDwL21osgJn7G3hks5tNkILgaJpZM4RlmOs.

itevomcid

[craigm-didata]

There you go. Thanks!

C

---

From: Bruno Moura [notifications@github.com] Sent: Tuesday, January 23, 2018 12:33 PM To: brunotm/elasticsplunk Cc: Craig Merchant (Americas); Author Subject: Re: [brunotm/elasticsplunk] ESS queries not working (#10)

This is the job inspector output, in the same window you’ll find a link for the search.log. Please attach it here.

Thanks

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<redir.aspx?REF=hN-k_n0580jZiOh_Ie0Goq3M0dol8KtAa72voZQelSACr4AswmLVCAFodHRwczovL3Byb3RlY3QtZXUubWltZWNhc3QuY29tL3MvWjhERkNSZ3BEc3ZXQXhPbHNORldNQw..>, or mute the thread<redir.aspx?REF=FINUCk2nYX0VHYaDrxFVWkfsZJenlFAzvV0qIKdILykCr4AswmLVCAFodHRwczovL3Byb3RlY3QtZXUubWltZWNhc3QuY29tL3MvT2lsa0NWbXdLQ3h5RFlaS0N5ZFk1Vw..>.

itevomcid

[brunotm]

We probably can handle it by updating the elasticsearch python library. I’ll check and update.

Thanks for investigating this! On Wed, 7 Feb 2018 at 16:32, vishnuSE notifications@github.com wrote:

Hi, This issue is because of elasticsearch version. ess query is not working with version elasticsearch 6.1.1. It would be really great if this issue can be fixed,

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/brunotm/elasticsplunk/issues/10#issuecomment-363827010, or mute the thread https://github.com/notifications/unsubscribe-auth/AGCQDrNrHV64OOwduN99RiCNPXT2FPXvks5tSdALgaJpZM4RlmOs .

-- Bruno Moura +351 932 518 676

[1vish99]

Hi Bruno, Please ignore my comment. i have downloaded the latest version of this app and it working fine with elasticsearch 6,1,1 as well. you need not apply any fix for the version change.

[brunotm]

Thanks! Closing!