Add some new fields to operator configuration sections to allow more flexible use of operators. This will open ThreatIngestor up to run multiple discrete tasks (e.g. send Twitter "open directory" results to a crawler, and send Twitter List c2 hits to ThreatKB) from a single instance and single config file.
Add support for the following fields:
[ ] allowed_sources: Comma-separated, whitespace-stripped, wildcard-supported list of source definitions (e.g. twitter-c2-list,rss-*). Only artifacts from these sources are sent to the operator.
[ ] artifact_conditions: Comma-separated, whitespace-stripped list of predefined conditions (e.g. disallow_ip that would only let through URL artifacts if they use a FQDN and not an IP address).
Additionally:
[ ] Add a new Conditions class, all conditions will have a function that can be passed an Artifact and return True or False. If True, the artifact will be processed, otherwise it will be skipped.
[ ] Document how to create Conditions classes to extend the tool, similar to how Source and Operator modules are described in the README.
Add some new fields to operator configuration sections to allow more flexible use of operators. This will open ThreatIngestor up to run multiple discrete tasks (e.g. send Twitter "open directory" results to a crawler, and send Twitter List c2 hits to ThreatKB) from a single instance and single config file.
Add support for the following fields:
allowed_sources
: Comma-separated, whitespace-stripped, wildcard-supported list of source definitions (e.g.twitter-c2-list,rss-*
). Only artifacts from these sources are sent to the operator.artifact_conditions
: Comma-separated, whitespace-stripped list of predefined conditions (e.g.disallow_ip
that would only let through URL artifacts if they use a FQDN and not an IP address).Additionally:
Conditions
class, all conditions will have a function that can be passed anArtifact
and returnTrue
orFalse
. IfTrue
, the artifact will be processed, otherwise it will be skipped.Conditions
classes to extend the tool, similar to howSource
andOperator
modules are described in theREADME
.