StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.
to: @airbnb/streamalert-maintainers
related to: #1250
resolves: #1265
Background
Unfortunately, Firehose Data Transformation is expensive, it charges us twice for the same data sent to a Firehose delivery stream. So we are moving the Artifact Extractor logic to Classifier, that will only increase Classifier lambda running time by few seconds.
Changes
Move artifact extractor logic to classifier, so the artifact extractor lambda function is deleted, as well as its related resources.
Add send_to_artifacts flag to normalizer, so we only send interesting information to artifacts table.
Update custom metrics for artifacts.
Update unit test cases.
Update Normalization docs.
Testing
Tested the new code in staging account and it is working as expected.
to: @airbnb/streamalert-maintainers related to: #1250 resolves: #1265
Background
Unfortunately, Firehose Data Transformation is expensive, it charges us twice for the same data sent to a Firehose delivery stream. So we are moving the
Artifact Extractorlogic to Classifier, that will only increase Classifier lambda running time by few seconds.Changes
send_to_artifactsflag to normalizer, so we only send interesting information toartifactstable.Normalizationdocs.Testing
Tested the new code in staging account and it is working as expected.