master dissertation : ELK stack+elastalert+X-Pack (elastalert doesn't send email)

[SadikiOnga]

Hello community, I'm actually working on my final project before graduating and the term is 'SIEM = ELKStack with elastalert for email alerting and X-Pack .

I've followed instruction given on a documentation, i can get match while testing rules. elastalert start normally but no email is received via gmail. Y I've configured SSMTP and can reciev emailvia command line. Honestly a big help is warmly welcomed. @Qmando

OS : CenTos 7 python v 2.7

### 1. rules ### 1.1 Example_rules.yaml****

name: Example frequency rule
type: frequency
index: filebeat-*
num_events: 2

timeframe:
  hours: 4

filter:
- query:
    query_string:
      query: "message: authentication failure OR failure password"
timestamp_field: "@timestamp"
alert:
- "email"
email: "xxxx@gmail.com"
smtp_host: "smtp.gmail.com" #for google email addresses
smtp_port: 587 #for google email addresses
smtp_ssl: true
from_addr: "elastalert@"
smtp_auth_file: "smtp_auth_file.yaml"

1.2 : smtp_auth_file.yaml

user: "pxxxxxxxx@gmail.com"
password: "xxxxxx"

1.3 : testing rules

Would have written the following documents to writeback index (default is elastalert_status):

silence - {'rule_name': 'Example frequency rule', '@timestamp': datetime.datetime(2019, 10, 3, 12, 59, 34, 674200, tzinfo=tzutc()), 'exponent': 0, 'until': datetime.datetime(2019, 10, 3, 13, 0, 34, 674193, tzinfo=tzutc())}

elastalert_status - {'hits': 1232, 'matches': 616, '@timestamp': datetime.datetime(2019, 10, 3, 12, 59, 34, 724094, tzinfo=tzutc()), 'rule_name': 'Example frequency rule', 'starttime': datetime.datetime(2019, 10, 3, 8, 57, 9, 275361, tzinfo=tzutc()), 'endtime': datetime.datetime(2019, 10, 3, 12, 59, 33, 275361, tzinfo=tzutc()), 'time_taken': 1.4450719356536865}

1.4 : starting elastalert

INFO:elastalert:Starting up
INFO:elastalert:Sleeping for 59.994702 seconds

2. ssmtp

mailhub=smtp.gmail.com:587   # SMTP server for Gmail
Hostname=xxx.xxx.tech
UseTLS=YES
UseSTARTTLS=Yes
FromLineOverride=YES #TO CHANGE FROM EMAIL
Root=pxxxxx@gmail.com # Redirect root email
AuthUser=xxxxxxxx@gmail.com
AuthPass=xxxxxxx
AuthMethod=LOGIN
RewriteDomain=gmail.com

file : revaliases

root:xxxxxxxx@gmail.com:smtp.gmail.com:587

### testing email email received **

mail -s "adasdas" xxxxxx@gmail.com

root <xxxxxxx@gmail.com> | root <xxxxxx@gmail.com> | 3:04 PM (1 minute ago) 
hello elastalert world

[SadikiOnga]

@Qmando I wish you can help on this or just a guidelines.

[Qmando]

Maybe try setting the from_addr to your full email address ? Google is probably rejecting the email because of that

[SadikiOnga]

@Qmando from_addr: "xxxxxx@gmail.com" as shown above, have changed it still email not received. Is there any other way of testing for email alerting, may be yahoo etc which is sure to receive email alerting

[nsano-rururu]

The error that occurred when specifying the Gmail address with from_addr was not a program problem but a setting problem. Launch a web browser on the terminal running ElastAlert, access the Google settings screen, go to the Google account → App that can access the account → Allow less secure apps: Disable → Change to enable. When I specified the Gmail address with from_addr and confirmed the operation, it worked normally.

smtp_port: 587

smtp_host: "smtp.gmail.com" 
smtp_port: 587 
smtp_ssl: false
smtp_auth_file: '/opt/elastalert/smtp/smtp_auth_user.yaml"

smtp_auth_user.yaml

user: xxx@gmail
password: xxx

smtp_port: 465

smtp_host: "smtp.gmail.com" 
smtp_port: 465 
smtp_ssl: true
smtp_auth_file: '/opt/elastalert/smtp/smtp_auth_user.yaml"

smtp_auth_user.yaml

user: xxx@gmail
password: xxx