search

elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
35 stars 448 forks source link

[FireEye Network Security]: update ingest pipeline to match upstream changes #11443

Open brightmatt opened 1 month ago

brightmatt commented 1 month ago

Integration Name

FireEye Network Security [fireeye]

Dataset Name

Unsure

Integration Version

1.22.0

Agent Version

8.12.2

Agent Output Type

elasticsearch

Elasticsearch Version

8.12.2

OS Version and Architecture

Cloud

Software/API Version

Cloud

Error Message

The error message indicates that the Elastic Agent is expecting a temp_ts field in the incoming logs, but this field is not present. The event.kind field showing “pipeline_error” suggests that there is an issue with the processing pipeline for the FireEye Network Security Integration, which is responsible for parsing and processing the incoming log data.

Event Original

<167>fenotify-33769.debug: CEF:0\|Trellix\|MPS\|10.0.1.997401\|IM\|infection-match\|1\|src=169.250.0.1 spt=10 smac=00:11:33:55:77:99 dst=127.0.0.20 dpt=20 dmac=00:22:44:66:88:aa dvchost=FireEye4500.gov.co.la-crosse.wi.us dvc=10.190.0.119 cn1Label=vlan cn1=0 cn2Label=sid cn2=30 cs1Label=sname cs1=Trellix-TestEvent-SIG-IM cs3Label=osinfo cs3=Trellix-TestEvent OS Info cs4Label=link cs4=https://FireEye4500.gov.co.la-crosse.wi.us/detection/objects?uuid\=cde9ccdc-28eb-4533-be93-872feb63f5cf rt=Oct 16 2024 21:31:09 GMT proto=tcp shost=IM-testing.fe-notify-examples.com externalId=33769 act=notified devicePayloadId=cde9ccdc-28eb-4533-be93-872feb63f5cf dvcmac=00:0B:AB:FF:9C:8E --

What did you do?

Image

What did you see?

in the event.kind field, it shows "pipline_error". The data from the alert does not show in the document.

What did you expect to see?

Correct data parsed out.

Anything else?

I contacted Elastic support and they confirmed that the intergration was configured correctly but noted that the data format is not coming over from the Fireye Network Appliance as expected. I reached out to Trellix support and they confirmed that the data format changed when they upgraded from version 9.1 to 10.0. See Trellix response below:

Trellix Response: Regarding your request, the notification format was changed starting from the 10.0 version as part of the rebranding process, so it's not possible to keep the same format as version 9.1. So, in this case, the SIEM provider will need to change their code to parse the logs. 

elasticmachine commented 1 month ago

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

efd6 commented 1 month ago

The ingest pipeline expects a JSON document, not a syslog message as you have shown in the issue. Have you configured the FireEye instance to send data as JSON? Does this still exist? Can Trellix provide information about the format that is being sent in v10?

The temp_ts field that you mention is not expected in the original document from FireEye, it is created in this pipeline from the rawmsg.timestamp field that is expected.

brightmatt commented 1 month ago

Thank you for the quick reply. The FireEye appliance has multiple formats to select, and I have tried them all. I switched it back to JSON and sent a test notification. See attached JSON document I got directly from the FireEye appliance and then event.original from Elastic.

I will reach out to Trellix to see if they can provide info on the format being sent.

<164>fenotify-33787.warning: { "appliance-id": "000BABFF9C8E", "product": "Web MPS", "version": "10.0.1.997401", "appliance": "FireEye4500.gov.co.la-crosse.wi.us", "msg": "normal", "alert": { "id": "33787", "name": "infection-match", "uuid": "c4d655da-75a0-4ed4-8576-33d679979366", "ack": "no", "sc-version": "1515.142", "severity": "minr", "explanation": { "analysis": "content", "protocol": "tcp", "malware-detected": { "malware": { "sid": "30", "name": "Trellix-TestEvent-SIG-IM", "stype": "bot-command" } } }, "src": { "vlan": "0", "ip": "169.250.0.1", "mac": "00:11:33:55:77:99", "port": "10", "host": "IM-testing.fe-notify-examples.com" }, "dst": { "ip": "127.0.0.20", "mac": "00:22:44:66:88:aa", "port": "20" }, "action": "notified", "alert-url": "https://FireEye4500.gov.co.la-crosse.wi.us/detection/objects?uuid=c4d655da-75a0-4ed4-8576-33d679979366", "occurred": "2024-10-17T01:16:17Z" } }

brightmatt commented 1 month ago

I forgot to include the list of formats that are available on the FireEye appliance.

[cid:d93828b7-05fd-49bc-8c3d-e7c0170657ca]

From: Matthew Bright @.> Sent: Wednesday, October 16, 2024 8:20 PM To: elastic/integrations @.>; elastic/integrations @.> Cc: Author @.> Subject: Re: [elastic/integrations] [FireEye Network Security]: update ingest pipeline to match upstream changes (Issue #11443)

Thank you for the quick reply. The FireEye appliance has multiple formats to select, and I have tried them all. I switched it back to JSON and sent a test notification. See attached JSON document I got directly from the FireEye appliance and then event.original from Elastic.

I will reach out to Trellix to see if they can provide info on the format being sent.

<164>fenotify-33787.warning: { "appliance-id": "000BABFF9C8E", "product": "Web MPS", "version": "10.0.1.997401", "appliance": "FireEye4500.gov.co.la-crosse.wi.us", "msg": "normal", "alert": { "id": "33787", "name": "infection-match", "uuid": "c4d655da-75a0-4ed4-8576-33d679979366", "ack": "no", "sc-version": "1515.142", "severity": "minr", "explanation": { "analysis": "content", "protocol": "tcp", "malware-detected": { "malware": { "sid": "30", "name": "Trellix-TestEvent-SIG-IM", "stype": "bot-command" } } }, "src": { "vlan": "0", "ip": "169.250.0.1", "mac": "00:11:33:55:77:99", "port": "10", "host": "IM-testing.fe-notify-examples.com" }, "dst": { "ip": "127.0.0.20", "mac": "00:22:44:66:88:aa", "port": "20" }, "action": "notified", "alert-url": "https://FireEye4500.gov.co.la-crosse.wi.us/detection/objects?uuid=c4d655da-75a0-4ed4-8576-33d679979366", "occurred": "2024-10-17T01:16:17Z" } } ________________________________ From: Dan Kortschak ***@***.***> Sent: Wednesday, October 16, 2024 7:19 PM To: elastic/integrations ***@***.***> Cc: Matthew Bright ***@***.***>; Author ***@***.***> Subject: Re: [elastic/integrations] [FireEye Network Security]: update ingest pipeline to match upstream changes (Issue #11443) The ingest pipeline expects a JSON document, not a syslog message as you have shown in the issue. Have you configured the FireEye instance to send data as JSON? Does this still exist? Can Trellix provide information about the format that is being sent in v10? The temp_ts field that you mention is not expected in the original document from FireEye, it is created in this pipeline from the rawmsg.timestamp field that is expected. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***> PRIVATE AND CONFIDENTIAL This e-mail and attachments are intended for the addressed recipient only. If you are not the correct recipient please notify the sender of the delivery error and delete this message. Improper disclosure, copying, distribution, retransmission, or use of information from this e-mail is Prohibited, and may result in liability and damages for misuse of this information.
brightmatt commented 1 month ago

Good evening,

I was doing more troubleshooting on my own and figured out I was configuring the SIEM integration in the wrong place on the FireEye appliance. My initial goal was to have the security alerts that are generated by the FireEye, like the ones below, to be sent to Elastic so that I could review them.

I found the correct spot in FireEye to enable the SIEM integration, but it only sends layer 7 data to Elastic and not the alerts.

My apologies for the confusion. Hoping the screenshot below can be added to your documentation to help prevent this confusion for other customers.

Thank you

[cid:0f4a367c-f396-4334-aaf8-7fa7edce6d93]

[cid:56aaa84a-7a4d-4320-9a4a-4c28eedf0118]

From: Matthew Bright @.> Sent: Wednesday, October 16, 2024 8:22 PM To: elastic/integrations @.>; elastic/integrations @.> Cc: Author @.> Subject: Re: [elastic/integrations] [FireEye Network Security]: update ingest pipeline to match upstream changes (Issue #11443)

I forgot to include the list of formats that are available on the FireEye appliance.

[cid:d93828b7-05fd-49bc-8c3d-e7c0170657ca]

From: Matthew Bright @.> Sent: Wednesday, October 16, 2024 8:20 PM To: elastic/integrations @.>; elastic/integrations @.> Cc: Author @.> Subject: Re: [elastic/integrations] [FireEye Network Security]: update ingest pipeline to match upstream changes (Issue #11443)

Thank you for the quick reply. The FireEye appliance has multiple formats to select, and I have tried them all. I switched it back to JSON and sent a test notification. See attached JSON document I got directly from the FireEye appliance and then event.original from Elastic.

I will reach out to Trellix to see if they can provide info on the format being sent.

<164>fenotify-33787.warning: { "appliance-id": "000BABFF9C8E", "product": "Web MPS", "version": "10.0.1.997401", "appliance": "FireEye4500.gov.co.la-crosse.wi.us", "msg": "normal", "alert": { "id": "33787", "name": "infection-match", "uuid": "c4d655da-75a0-4ed4-8576-33d679979366", "ack": "no", "sc-version": "1515.142", "severity": "minr", "explanation": { "analysis": "content", "protocol": "tcp", "malware-detected": { "malware": { "sid": "30", "name": "Trellix-TestEvent-SIG-IM", "stype": "bot-command" } } }, "src": { "vlan": "0", "ip": "169.250.0.1", "mac": "00:11:33:55:77:99", "port": "10", "host": "IM-testing.fe-notify-examples.com" }, "dst": { "ip": "127.0.0.20", "mac": "00:22:44:66:88:aa", "port": "20" }, "action": "notified", "alert-url": "https://FireEye4500.gov.co.la-crosse.wi.us/detection/objects?uuid=c4d655da-75a0-4ed4-8576-33d679979366", "occurred": "2024-10-17T01:16:17Z" } } ________________________________ From: Dan Kortschak ***@***.***> Sent: Wednesday, October 16, 2024 7:19 PM To: elastic/integrations ***@***.***> Cc: Matthew Bright ***@***.***>; Author ***@***.***> Subject: Re: [elastic/integrations] [FireEye Network Security]: update ingest pipeline to match upstream changes (Issue #11443) The ingest pipeline expects a JSON document, not a syslog message as you have shown in the issue. Have you configured the FireEye instance to send data as JSON? Does this still exist? Can Trellix provide information about the format that is being sent in v10? The temp_ts field that you mention is not expected in the original document from FireEye, it is created in this pipeline from the rawmsg.timestamp field that is expected. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***> PRIVATE AND CONFIDENTIAL This e-mail and attachments are intended for the addressed recipient only. If you are not the correct recipient please notify the sender of the delivery error and delete this message. Improper disclosure, copying, distribution, retransmission, or use of information from this e-mail is Prohibited, and may result in liability and damages for misuse of this information.
efd6 commented 1 month ago

@brightmatt Can you repost the information that you sent by email as a comment from the GH UI? Sending by email does not result in properly formatted comments. In particular the last comment where you say there is an image. I have tried to repair the other comments, but some attribute is set when an email is the origin of a comment which causes all the formatting to be broken.