search

elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
30 stars 447 forks source link

[ti_custom]: 406 Not Acceptable - Integration won't return any indicators #11324

Open nicpenning opened 1 month ago

nicpenning commented 1 month ago

Integration Name

Custom Threat Intelligence [ti_custom]

Dataset Name

ti_custom.indicator

Integration Version

0.2.0

Agent Version

8.15.2

Agent Output Type

elasticsearch

Elasticsearch Version

8.15.2

OS Version and Architecture

Windows 2019

Software/API Version

STIX/TAXII

Error Message

GET:406 Not Acceptable (406)

Event Original

No events are getting returned

What did you do?

Installed the 0.2.0 integration and populated the configuration fields the best that we could with username/password/urls, etc..

What did you see?

We see this message in Elastic for the result:

"message": [
        "GET:406 Not Acceptable (406)",
        "Processor \"conditional\" with tag \"\" in pipeline \"logs-ti_custom.indicator-0.2.0\" failed with message \"Error during CEL program evaluation\"\n"
      ]

What did you expect to see?

I expect that the integration will successfully authenticate and receive indicators from STIX/TAXII feed (MS-ISAC/EI-ISAC/Auto-ISAC/etc)

Anything else?

It seems like the initial interval doesn't seem to work. entering 72h or 30s, we receive a 404 message.

Our theory is that there is too much information being returned from the server and it simply gives us the 406 message.

We know that the URL/Creds work when using raw/native python modules to do this.

nicpenning commented 1 month ago

Tested this shortly after this change: https://github.com/elastic/integrations/pull/11202

elasticmachine commented 1 month ago

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

efd6 commented 1 month ago

@nicpenning This looks related https://github.com/mitre/cti/issues/104. What Accept header do you use with the python code? The integration uses "application/taxii+json;version=2.1". If they differ, perhaps this should be configurable.

nicpenning commented 1 month ago

We will reach out to our source and compare our manual pull to this header.