search

elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
187 stars 397 forks source link

[Meta][CrowdStrike] Supporting Event Streaming API #9082

Open jamiehynds opened 5 months ago

jamiehynds commented 5 months ago

Our existing CrowdStrike integration requires the Falcon SIEM Connector in order to pull detections and audit events from Falcon. This adds complexity and a maintenance overhead for users which could be eliminated if our integration connected directly to Falcon's Event Streaming API to pull the data.

The Event Streaming API works differently to most API's we interact with and our CEL or HTTPJSON inputs cannot currently handle API. In order to get data, you keep one request open indefinitely, while periodically making an independent request to refresh the streaming session to keep it alive.

In order to unblock us from adding support for the Event Streaming API, we need to research if a new input is required or can we bridge the functionality gap via one of our existing inputs. 

### Tasks
- [ ] https://github.com/elastic/integrations/issues/10262
elasticmachine commented 5 months ago

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

mbudge commented 5 months ago

"complexity and a maintenance overhead for users"

We've had it running on a server for 2-3 years with no issues. It handles all the log rotation so it's very easy to maintain.

jamiehynds commented 5 months ago

Thanks for the feedback @mbudge. Based on that, if we supported the Event Streaming API directly would you envisage some value there or likely to remain on SIEM Connector?

mbudge commented 5 months ago

The advantage of using the SIEM connector is you can push customers to use crowdstrike support if there is a problem downloading data.

As this one is a bit complicated with persistent connections I'd prefer to use the SIEM connector.

NateUT99 commented 5 months ago

We have used the event stream API with a few other products, and have had good success with it. I would certainly be comfortable moving our production data ingest to this (from SIEM connector) once it becomes available.

ShourieG commented 2 weeks ago

The last time we had discussions on supporting event stream inputs the following approaches were decided upon :-

@brijesh-elastic @piyush-elastic, I recall last time we spoke on this, support for CEL was requested because support for response manipulation was required as well as support for URL modification based on cursor values. Could you confirm this and expand on the reason for CEL support ?

cc: @efd6

piyushw-crest commented 2 weeks ago

@ShourieG - Yes we did discuss on this last time for CrowdStrike only, as we have option to configure event stream API and access it via http endpoint. For more details refer this pdf. Let me know if you need more help on the same.

ShourieG commented 2 weeks ago

@ShourieG - Yes we did discuss on this last time for CrowdStrike only, as we have option to configure event stream API and access it via http endpoint. For more details refer this pdf. Let me know if you need more help on the same.

@piyush-elastic thanks for the update, I was curious on the CEL support. Do you think it would add value and if what would be the use case here?