CrowdStrike / falcon-orchestrator

CrowdStrike Falcon Orchestrator provides automated workflow and response capabilities
GNU Affero General Public License v3.0
185 stars 60 forks source link

Error occured while trying to save detection event to database #25

Open cuongdinh opened 7 years ago

cuongdinh commented 7 years ago

Hello,

We have a detection that Falcon Orchestrator gets stuck on, looks like it couldn't save it and keeps sending us the detection email. Also, the Falcon Orchestrator Client service doesn't seem to run. I keep trying to start it and seconds later it stops. Any advice? Thanks!

2017-03-01 16:50:41,734 DEBUG FalconOrchestrator.Client.FalconOrchestratorService - Connection to database is successful, starting service 2017-03-01 16:50:46,128 DEBUG FalconOrchestrator.Client.FalconOrchestratorService - [21002] Event already stored in database 2017-03-01 16:50:46,238 DEBUG FalconOrchestrator.Client.Rule - Resolved IP address of xx.xx.xx.xx for host 2017-03-01 16:50:46,269 DEBUG FalconOrchestrator.Client.Rule - Notification rule is enabled and severity of High is above threshold, attempting to send email 2017-03-01 16:50:46,707 FATAL FalconOrchestrator.Client.EventModel - [21003] Error occured while trying to save detection event to database System.Data.Entity.Validation.DbEntityValidationException: Validation failed for one or more entities. See 'EntityValidationErrors' property for more details. at System.Data.Entity.Internal.InternalContext.SaveChanges() at System.Data.Entity.Internal.LazyInternalContext.SaveChanges() at System.Data.Entity.DbContext.SaveChanges() at FalconOrchestrator.Client.Persistence.SaveToDatabase(String cid, String offset) at FalconOrchestrator.Client.DetectionModel.Save()

mr-burnse commented 7 years ago

Hey @cuongdinh I got some of the details from your colleague and confirmed this is related to issue #6 . I'm also going to file this under a feature request as a way to allow the user to define what sure occur under this condition (fail and stop the service or skip over the event).

A workaround solution is to skip over the problematic event by setting the offset to n+1 (21004 in this case) within the admin > configuration section in the UI. I've provided your colleague with steps on how to fix this issue within the code/DB as well. If you're still running into issues let me know!

cuongdinh commented 7 years ago

Thanks! I got your email from my colleague and the workaround works for us. Thanks for your help!

mendoza2fast commented 7 years ago

I think we are hitting this one also. Incremented the key copying over the patched client has not resolved it in this case...

2017-06-14 20:40:13,117 DEBUG FalconOrchestrator.Client.FalconOrchestratorService - Connection to database is successful, starting service 2017-06-14 20:40:20,366 FATAL FalconOrchestrator.Client.FalconOrchestratorService - Error saving detection event to database: Validation failed for one or more entities. See 'EntityValidationErrors' property for more details. The validation errors are: The field Description must be a string or array type with a maximum length of '200'.

offset: 16172 2017-06-14 21:14:02,717 DEBUG FalconOrchestrator.Client.FalconOrchestratorService - Connection to database is successful, starting service 2017-06-14 21:14:10,425 DEBUG FalconOrchestrator.Client.EventModel - [16180] Authentication audit event saved to database 2017-06-14 21:14:10,440 DEBUG FalconOrchestrator.Client.EventModel - [16181] Authentication audit event saved to database 2017-06-14 21:14:10,456 DEBUG FalconOrchestrator.Client.EventModel - [16182] Authentication audit event saved to database 2017-06-14 21:14:10,456 DEBUG FalconOrchestrator.Client.EventModel - [16183] Authentication audit event saved to database 2017-06-14 21:14:10,472 DEBUG FalconOrchestrator.Client.EventModel - [16184] Authentication audit event saved to database 2017-06-14 21:14:10,472 DEBUG FalconOrchestrator.Client.EventModel - [16185] Authentication audit event saved to database 2017-06-14 21:14:10,487 DEBUG FalconOrchestrator.Client.EventModel - [16186] Authentication audit event saved to database 2017-06-14 21:14:10,487 DEBUG FalconOrchestrator.Client.EventModel - [16187] Authentication audit event saved to database 2017-06-14 21:14:10,503 DEBUG FalconOrchestrator.Client.EventModel - [16188] Authentication audit event saved to database 2017-06-14 21:14:10,503 DEBUG FalconOrchestrator.Client.EventModel - [16189] Authentication audit event saved to database 2017-06-14 21:14:10,518 DEBUG FalconOrchestrator.Client.EventModel - [16190] Authentication audit event saved to database 2017-06-14 21:14:10,518 DEBUG FalconOrchestrator.Client.EventModel - [16191] Authentication audit event saved to database 2017-06-14 21:14:10,534 DEBUG FalconOrchestrator.Client.EventModel - [16192] Authentication audit event saved to database 2017-06-14 21:14:10,550 DEBUG FalconOrchestrator.Client.EventModel - [16193] Authentication audit event saved to database 2017-06-14 21:14:10,550 DEBUG FalconOrchestrator.Client.EventModel - [16194] Authentication audit event saved to database 2017-06-14 21:14:10,550 DEBUG FalconOrchestrator.Client.EventModel - [16195] Authentication audit event saved to database 2017-06-14 21:14:10,565 DEBUG FalconOrchestrator.Client.EventModel - [16196] Authentication audit event saved to database 2017-06-14 21:14:10,565 DEBUG FalconOrchestrator.Client.EventModel - [16197] Authentication audit event saved to database 2017-06-14 21:14:10,581 DEBUG FalconOrchestrator.Client.EventModel - [16198] Authentication audit event saved to database 2017-06-14 21:14:10,581 DEBUG FalconOrchestrator.Client.EventModel - [16199] Authentication audit event saved to database 2017-06-14 21:14:10,597 DEBUG FalconOrchestrator.Client.EventModel - [16200] Authentication audit event saved to database 2017-06-14 21:14:10,597 DEBUG FalconOrchestrator.Client.EventModel - [16201] Authentication audit event saved to database 2017-06-14 21:14:10,612 DEBUG FalconOrchestrator.Client.EventModel - [16202] Authentication audit event saved to database 2017-06-14 21:14:10,612 DEBUG FalconOrchestrator.Client.EventModel - [16203] Authentication audit event saved to database 2017-06-14 21:14:10,628 DEBUG FalconOrchestrator.Client.EventModel - [16204] Authentication audit event saved to database 2017-06-14 21:14:10,722 FATAL FalconOrchestrator.Client.FalconOrchestratorService - Error saving detection event to database: Validation failed for one or more entities. See 'EntityValidationErrors' property for more details. The validation errors are: The field Description must be a string or array type with a maximum length of '200'. offset: 16205
mr-burnse commented 7 years ago

@mendoza2fast Can you join the slack chat room (link on the main project page). I will provide you a fix on there.

copos45 commented 7 years ago

Hello Evan, I'm running into the same issue as mendoza2fast. I've tried changing the offset to a higher value and restarting the falcon orchestrator service but the debug log is consistently reporting: 2017-09-14 09:33:24,022 FATAL FalconOrchestrator.Client.FalconOrchestratorService - Error saving detection event to database: Validation failed for one or more entities. See 'EntityValidationErrors' property for more details. The validation errors are: The field Description must be a string or array type with a maximum length of '200'. The event in question far exceeds the 200/500 char limit and is actually 1762 char so it would be a good idea to change the value to 2500char and allow such large events to be bypassed in whatever fix is released. thank you.

mr-burnse commented 7 years ago

@copos45 - 1762 characters? Yikes, didn't expect it to ever get that big. Can you send me the description for the event that is triggering this? I agree though, will bump the character limit up substantially in the next build. In the interim if you need a workaround, ping me on a slack and I can provide a fix.

copos45 commented 7 years ago

please note my email address change Sure, I’ll send the description Monday and we can discuss it further then. Thank you, Christopher Chan

From: Evan Burns [mailto:notifications@github.com] Sent: Friday, September 15, 2017 11:04 AM To: CrowdStrike/falcon-orchestrator Cc: Chan, Christopher; Mention Subject: Re: [CrowdStrike/falcon-orchestrator] Error occured while trying to save detection event to database (#25)

@copos45https://github.com/copos45 - 1762 characters? Yikes, didn't expect it to ever get that big. Can you send me the description for the event that is triggering this? I agree though, will bump the character limit up substantially in the next build. In the interim if you need a workaround, ping me on a slack and I can provide a fix.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/CrowdStrike/falcon-orchestrator/issues/25#issuecomment-329808753, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AbzUQSkbT3AqirsaZg1E_3o0ICUJYVs2ks5sipHBgaJpZM4MQXt0.

This e-mail, including any attachments, may be confidential, privileged, or otherwise legally protected. It is intended only for the addressee. If you received this e-mail in error or from someone who was not authorized to send it to you, do not disseminate, copy, or otherwise use this e-mail or its attachments. Please notify the sender immediately by reply e-mail and delete the e-mail from your system.

dlkeeling commented 6 years ago

Hi Evan,

I have the same issue, I have tried setting the offset to 21004 but the issue persists.

copos45 commented 6 years ago

please note my email address change This worked for me:

start\run\cmd sqlcmd use FalconOrchestrator alter table Detections alter column Description nvarchar(500) null exit

and see the June.27, 2017 entry-> https://falcon-orchestrator.slack.com/messages/C20SJ71AQ/search/evan%20burns/ replace the FalconOrchestrator.DAL.dll with the one i supplied here

From: dlkeeling [mailto:notifications@github.com] Sent: Thursday, October 12, 2017 2:42 PM To: CrowdStrike/falcon-orchestrator Cc: Chan, Christopher; Mention Subject: Re: [CrowdStrike/falcon-orchestrator] Error occured while trying to save detection event to database (#25)

Hi Evan,

I have the same issue, I have tried setting the offset to 12004 but the issue persists.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/CrowdStrike/falcon-orchestrator/issues/25#issuecomment-336229113, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AbzUQZE9joh7WY4tmIy7aroDclMslRMrks5srl2PgaJpZM4MQXt0.

This e-mail, including any attachments, may be confidential, privileged, or otherwise legally protected. It is intended only for the addressee. If you received this e-mail in error or from someone who was not authorized to send it to you, do not disseminate, copy, or otherwise use this e-mail or its attachments. Please notify the sender immediately by reply e-mail and delete the e-mail from your system.