General problems

[devatnull]

Hello joshua,

I tried the commits, removed slack, twimlet etc, changed sms alert functions to add to 60min lists and dicts instead of 30min's

But there is a problem with it, it alerts old alerts on the hive, not recent alerts that are 30 mins old or 60 mins old. another problem is that if I have it set to query severity based on 3, but lets say my defined critical word based alert is a medium alert. The script will not alert the defined word since it is a medium alert.

I really need to do the following: 1- Query severity high alerts and alert after when it hit hives 2 minutes later 2- Find for words in severity 2 alerts but only words not the rest 3- Alert in 2 minutes after the alert has been found only via sms

[JoshuaSmeda]

Hi @devatnull, can you zip up your config and send it to me - it makes it easier for me to read it instead of following the snippets?

You're missing an important portion of the code as well that makes the 2-minute alert work, without it I can't help much.

Can you do the above for me?

[JoshuaSmeda]

Thanks, @devatnull, I'll check this out and let you know.

Just to be sure, have you tried checkout the master branch instead of copying snippets of code into your project? The master branch has the working functionality you seek.

9 has a bunch of new features/improvements that I'm hoping will be released by this weekend

[devatnull]

this issue is based on latest master branch

[devatnull]

I really need to do the following: 1- Query severity 3 alerts and alert after when it hit hives 2 minutes later 2- Find for critical words in severity 2 alerts while the script is set to query on severity3 and alert the critical word even if it is severity2-1-3-4 3- Alert in 2 minutes after the alert has been found only via sms

[devatnull]

Hello joshua, Any suggestions?

[JoshuaSmeda]

Hello @devatnull,

I've added the support for #9 which should address your problems above. Please checkout js_issue9.

You don't need to edit anything in the application besides configuration.py

For example:

    'THEHIVE_LEVEL3': {'ENABLED': True,
                       'LOW_SEVERITY': {'TIMER': 1800, 'NOTIFICATION_METHOD': ['SLACK_API']},
                       'MEDIUM_SEVERITY': {'TIMER': 2700, 'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_SMS']},
                       'HIGH_SEVERITY': {'TIMER': 3600, 'NOTIFICATION_METHOD': ['TWILIO_CALL']},
                       'HIGH_RISK': {'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_CALL']}}

You can enable alert checks for the 3 different Hive levels, and then say what you want to happen when the different tiers get reached based on their alert age. You can also adjust the alert method for high_risk words alert

SYSTEM_SETTINGS = {
    'HIGH_RISK_WORDS': ['HIGH_RISK', 'Critical'],
    'HIGH_RISK_WORDS_SEVERITY_LEVEL': 2,

Adjust the severity level you want to get high_risk_words for and the different words to check for. Remember, it only looks at the title or the artifacts.

Let me know if it works so I can get it merged to production if you're happy.

[devatnull]

Hello Joshua,

Looks perfect, cutting long character messages is also working very well except lets say when there is 10 alerts that needs to be sent at the same time, it will randomly send them hence, we won't notice which half of the message belongs to which half. Maybe this could be achieved with keeping the sent message in a function and make it append a message id on the msg_body and then append it to the same message then repeat. What do you think?

[JoshuaSmeda]

Thanks for testing, happy to hear that it looks good. The jumbled SMS's is due to Twilio not following the order in which they are sent to their API (or just delays at a cellular network level for example). I will add the last 5 digits of the message ID to the SMS, which should be unique and not too confusing.

On Wed, Feb 3, 2021 at 3:41 PM devatnull notifications@github.com wrote:

Hello Joshua,

Looks perfect, cutting long character messages is also working very well except lets say when there is 10 alerts that needs to be sent at the same time, it will randomly send them hence, we won't notice which half of the message belongs to which half. Maybe this could be achieved with keeping the sent message in a function and make it append a message id on the msg_body and then append it to the same message then repeat. What do you think?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JoshuaSmeda/thehive_sla_monitor/issues/11#issuecomment-772513857, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJRBWDSP2GNUL3QW7GTIPLLS5FG7XANCNFSM4WHBGHLQ .

[devatnull]

Hello Joshua,

I will add the last 5 digits of the message ID to the SMS, which should be unique and not too confusing.

What about a queue mechanism to send them one by one based on the ID that you append?

There is another problem here, which is:

After starting main.py it will query even 6 days old alerts and pushes them to TWILIO_SMS, we have so many alerts that if I keep it running more than 5 seconds we will finish our twilio quota. Any suggestions on this? it should only query the defined severity but alert if it is 10 minutes older. Maybe in your tests you missed because of version difference?

Elastic version is: "number" : "5.6.16", "lucene_version" : "6.6.1" TheHive version: 3.4.2-1

[devatnull]

After starting main.py it will query even 6 days old alerts and pushes them to TWILIO_SMS, we have so many alerts that if I keep it running more than 5 seconds we will finish our twilio quota. Any suggestions on this? it should only query the defined severity but alert if it is 10 minutes older. Maybe in your tests you missed because of version difference?

Evidence:

2021-02-03 16:46:30,881 [INFO] [*] TheHive Alert: Title: Some Hosts are Down. (ff7554f41a0ac35c9b7f6fb68b329347). Created at 2021-01-28 16:15:37.

2021-02-03 16:46:30,881 [WARNING] HIGH Severity Breach (ff7554f41a0ac35c9b7f6fb68b329347). Alert Age: 6 days, 0:30:53
Nothing to change.

2021-02-03 16:46:30,881 [INFO] Total message character size: 125

2021-02-03 16:46:30,929 [INFO] POST Request: https://api.twilio.com/2010-04-01/Accounts/REDACTED/Messages.json

2021-02-03 16:46:30,929 [INFO] PAYLOAD: {'To': '+222222222', 'From': '+1111111111', 'Body': 'TheHive SLA Breach - ff7554f41a0ac35c9b7f6fb68b329347.\nAdditional Information: Some Hosts are Down.'}

2021-02-03 16:46:31,562 [INFO] POST Response: 201 {"sid": "REDACTED_SMS_ID", "date_created": "Wed, 03 Feb 2021 13:46:31 +0000", "date_updated": "Wed, 03 Feb 2021 13:46:31 +0000", "date_sent": null, "account_sid": "REDACTED", "to": "+222222222", "from": "+1111111111", "messaging_service_sid": null, "body": "TheHive SLA Breach - ff7554f41a0ac35c9b7f6fb68b329347.\nAdditional Information: Some Hosts are Down.", "status": "queued", "num_segments": "1", "num_media": "0", "direction": "outbound-api", "api_version": "2010-04-01", "price": null, "price_unit": "USD", "error_code": null, "error_message": null, "uri": "/2010-04-01/Accounts/REDACTED/Messages/REDACTED_SMS_ID.json", "subresource_uris": {"media": "/2010-04-01/Accounts/REDACTED/Messages/REDACTED_SMS_ID/Media.json"}}

2021-02-03 16:46:31,562 [INFO] SMS Sent: REDACTED_SMS_ID

Current configuration:

SLA_SETTINGS = {
    'THEHIVE_LEVEL3': {'ENABLED': True,
                       'LOW_SEVERITY': {'TIMER': 1800, 'NOTIFICATION_METHOD': ['TWILIO_SMS']},
                       'MEDIUM_SEVERITY': {'TIMER': 2700, 'NOTIFICATION_METHOD': ['TWILIO_SMS']},
                       'HIGH_SEVERITY': {'TIMER': 3600, 'NOTIFICATION_METHOD': ['TWILIO_SMS']},
                       'HIGH_RISK': {'NOTIFICATION_METHOD': ['TWILIO_SMS']}},

    'THEHIVE_LEVEL2': {'ENABLED': False,
                       'LOW_SEVERITY': {'TIMER': 1800, 'NOTIFICATION_METHOD': ['TWILIO_SMS']},
                       'MEDIUM_SEVERITY': {'TIMER': 2700, 'NOTIFICATION_METHOD': ['TWILIO_SMS']},
                       'HIGH_SEVERITY': {'TIMER': 3600, 'NOTIFICATION_METHOD': ['TWILIO_SMS']},
                       'HIGH_RISK': {'NOTIFICATION_METHOD': ['TWILIO_SMS']}},

    'THEHIVE_LEVEL1': {'ENABLED': False,
                       'LOW_SEVERITY': {'TIMER': 1800, 'NOTIFICATION_METHOD': ['SLACK_API']},
                       'MEDIUM_SEVERITY': {'TIMER': 2700, 'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_SMS']},
                       'HIGH_SEVERITY': {'TIMER': 3600, 'NOTIFICATION_METHOD': ['TWILIO_CALL']},
                       'HIGH_RISK': {'NOTIFICATION_METHOD': ['SLACK_API', 'TWILIO_CALL']}}
}

SYSTEM_SETTINGS = {
    'HIGH_RISK_WORDS': ['SUBAT'],
    'HIGH_RISK_WORDS_SEVERITY_LEVEL': 2,
    'HIVE_SERVER_IP': '192.168.122.30',
    'HIVE_SERVER_PORT': 9000,
    'HIVE_FQDN': 'http://192.168.122.30',
    'HIVE_API_KEY': '*****',
    'LOG_FILE_LOCATION': 'debug.log'
}

FLASK_SETTINGS = {
    'ENABLE_WEBSERVER': True,
    'FLASK_WEBSERVER_IP': '192.168.122.30',
    'FLASK_WEBSERVER_PORT': 3002
}

TWILIO_SETTINGS = {
    'TWILIO_ENABLED': True,
    'TWILIO_SENDER': '+***********',
    'TWILIO_RTCP': ['********'],
    'ACCOUNT_SID': '**************',
    'AUTH_TOKEN': '****************',
    'TWIMLET_URL': ''
}

SLACK_SETTINGS = {
    'SLACK_ENABLED': False,
    'SLACK_APP_TOKEN': '',
    'SLACK_CHANNEL': '',
    'SLACK_WEBHOOK_URL': ''
}

[JoshuaSmeda]

A queue mechanism won't be necessary since I already send messages in the order that its detected. It's Twilio that messes up the order.

After starting main.py it will query even 6 days old alerts - this is what the app was designed to do. You shouldn't keep alerts open for so long, since they should be imported as cases and attended to. So basically any alert older than the high_severity timer - it'll trigger the alerts and re-trigger again if you restart the application (since we don't store data persistently, i.e in a database).

It has always been this way since day 1.

Alternatively, I add a max_age timer of say, 6 days, and any alerts that is older than 6 days, we'll ignore? I'll make the option configurable to your use case in configuration.py. Let me know if that will work for you?

[devatnull]

I understand the use case that this was designed, but our use case is more about system administration alerts for the system admins to get notified via sms, we don't have dedicated personnel for these tickets. As I understand it will not alerts earlier than 3600 seconds but it will alert the older ones. Another option which alerts messages that are no older than 3600 seconds.

[JoshuaSmeda]

Understood, will this work for you then?

I can add a max_age timer of say, 6 days, and any alerts that is older than 6 days, we'll ignore? I'll make the option configurable to your use case in configuration.py. This is a risky change and you would just need to get the timers right so you don't accidentally miss important alerts that didn't fire because they're older than the max_age timer.

[devatnull]

that would be perfect. Thank you.

[JoshuaSmeda]

Perfect will branch this off into a new request for easier tracking.