Alert on tampering with app logs

[mogul]

In order to ensure we can vouch for the integrity of application logs, we want any activity which might imply they've been compromised to result in alerts.

Acceptance Criteria

• [ ] When someone tries to UPDATE or DELETE a log entry via the ElasticSearch port, an alert is generated
• [ ] When someone other than ElasticSearch tries to modify the ElasticSearch data store (eg a different user on that host), an alert is generated

Implementation sketch

• use HAproxy in front of ElasticSearch to block update/delete requests
• use auditd logs to look for non-ES users touching the files

[sharms]

Completed initial research and are prototyping solution. ETA 2 days

[mogul]

Works in staging, tested as sending alerts... Just needs deployment, which will happen today.

[sharms]

Confirmed these are in production as of Sep 14