Code is GPLv3, (c) Arnim Eijkhoudt, 2022-2024.
Code probably has bugs, but it is officially in a 'works for me' and 'works for others' state ;-)
MatterBot consists of two parts that can be run independently: matterbot and matterfeed. Both parts should be run within tmux or screen; the code itself is not daemonized (this may happen at some point in the future).
matterfeed
SourcesMatterfeed reports news updates on a set schedule. The currently supported sources are listed in the table below:
Name | Type | API Key Required | Paid Subscription |
---|---|---|---|
Aqua Security Blog | RSS | No | No |
Bruce Schneier's Blog | RSS | No | No |
CISecurity | RSS | No | No |
Cqure Blog | RSS | No | No |
CSHub (configurable list of CSHub feeds) | RSS | No | No |
DarkNet blog | RSS | No | No |
DataBreachSecurity News | RSS | No | No |
DIVD CSIRT | RSS | No | No |
GBHackers News | RSS | No | No |
Group-IB Blog | RSS | No | No |
Kitploit Tool Updates | RSS | No | No |
KnowBe4 News | RSS | No | No |
KrebsOnSecurity Blog | RSS | No | No |
MajorLeagueHacking News | RSS | No | No |
Microsoft Vulnerability Reports | RSS | No | No |
NCSC Netherlands Advisories | RSS | No | No |
NCSC United Kingdom Advisories | RSS | No | No |
OpenCVE feed of CVEs | RSS | Yes | No |
PacketStorm Security | RSS | No | No |
Ransomwatch | JSON | No | No |
Reddit (configurable list of subreddits) | RSS | No | No |
SANS Internet Storm Center | RSS | No | No |
SebDraven | RSS | No | No |
SecureList News | RSS | No | No |
SecurityAffairs News | RSS | No | No |
Spiceworks Tech News | RSS | No | No |
TheHackerNews News | RSS | No | No |
The Record Media | RSS | No | No |
Threatpost News | RSS | No | No |
TrendMicro Research | RSS | No | No |
Tripwire State of Security | RSS | No | No |
US-CERT National Cyber Awareness System (Advisories, Alerts, Analysis Reports, Current Activity) | RSS | No | No |
Velociraptor News/Updates | RSS | No | No |
WeLiveSecurity News | RSS | No | No |
WikiJS Page Updates | WikiJS GraphQL | Yes | No |
New Matterfeed modules can be created. A boilerplate example can be found in the modules
directory.
matterbot
CommandsThe Matterbot component listens in a given set of channels (configurable per module) for user-definable commands, executes and returns the results of the module code. The currently supported commands are listed below:
Name | Type | Functionality / Use Case | API Key Required | Paid Subscription |
---|---|---|---|---|
AbuseIPDB | Threat Intel | Look up IPv4, IPv6 and netblocks for indicators/reports of abuse | Yes | No, but higher tier API rate limits are only available to paid subscribers |
AlienVault OTX | Threat Intel | Look up IPv4, IPv6, hostnames, domains, MD5/SHA1/SHA256 hashes and URLs | No | No, but some API limitations may apply |
ASN WHOIS | Threat Intel | Look up Autonomous System Numbers and return the ownership, peering and location information | No | No |
AttackMatrix | Threat Intel | Query an AttackMatrix instance for e.g. MITRE ATT&CK IDs, Actor- and TTP-overlap. Requires Python GraphViz bindings to display the accompanying Graph | No, unless the AttackMatrix API is configured to require an API key | No |
Bootloaders | Threat Intel | Query 'Bootloaders' project for vulnerable/malicious bootloader information. Returns detailed information, hashes and detection rules | No | No |
Broadcom Symantec Security Cloud (BSSC) | Threat Intel | Retrieve 'Threat Intel Insight' information for SHA256 file hashes, IPs, reputations, domains and URLs | Yes | Yes |
Censys | Threat Intel | Query Censys for IPs and SHA256 certificate fingerprints. Query results are returned as the original Censys JSON blob | Yes | No: basic functionality Yes: additional features, such as pagination |
ChatGPT | LLM / GPT queries | Ask OpenAI's ChatGPT singular questions (no support for chat history). Requires a paid subscription with sufficient credits | Yes | Yes |
Diceroll | Fun | Roll any kind of dice combination: #d# format | No | No |
DocGen | Documentation | Automatically create documentation with templated variables, rendering and more. Probably useless unless you have all the required materials. | No | No |
Early Warning & Advisory (EWA) | Threat Intel | Create Early Warning & Advisory documents using the National Vulnerability Database (NVD) and WikiJS information. Requires pandoc, pypandoc, LaTeX, a WikiJS instance and a CSS template for final rendering | No (for NVD) Yes (for WikiJS) |
No |
GeoLocation | Threat Intel | Convert latitude/longitude values into an address | No | No |
GreyNoise | Threat Intel | Query the GreyNoise API for IP address reputation, such as whether an IP has been observed scanning the internet, source & destination countries, fingerprints, ports scanned, whether it is benign or not, etc. | Yes | Yes: certain features require an additional subscription license, such as timeline and similarity features; see the GreyNoise website and API documentation or more information |
GTFOBins | Threat Intel | Query the *' |
No | No |
Hybrid-Analysis | Threat Intel | Look up IPs, hostnames, domains, URLs, MD5, SHA1, SHA256, Authentihash, Imphash, ssdeep hashes and VxFamily names, as well as known 'context' and 'similarity' | Yes: 'vetted' API key strongly recommended to prevent hitting API limits | No: basic functionality, Yes: additional features/details |
IPLocation | Threat Intel | Look up general location information (country, ISP) for an IPv4 of IPv6 address | No | No |
IPWHOIS | Threat Intel | Look up IP address information: ownership, ASN, geolocation information | No | No |
LeakIX | Threat Intel | Find subdomains and look up possible information/data leaks for hosts and domains | Yes: API key strongly recommended to prevent hitting API limits | No: basic functionality, Yes: additional data |
LOLBAS | Threat Intel | Query the 'Living Off The Land Binaries, Scripts and Libraries' project for file information. Returns detailed information and detection rules | No | No |
LOLDrivers | Threat Intel | Query 'Living Off The Land Drivers' project for driver information. Returns detailed information, hashes and detection rules | No | No |
Malpedia | Threat Intel | Look up malware families, threat actors and MD5/SHA256 malware hashes | No: basic functionality Yes: malware downloads |
No |
MalwareBazaar | Threat Intel | Query MalwareBazaar for MD5/SHA1/SHA256 hashes of malware. Will also return include a downloadable malware sample, if available | No | No |
MISP | Threat Intel | Wildcard-searching of a MISP instance for the given search terms. Returns links to the MISP Events where the search terms have been found | Yes | No |
OpenCVE | Vulnerability Management | Query an OpenCVE instance for information about vulnerabilities. | Yes | No |
Qualys | Vulnerability Management | Query the Qualys CSAM API for software/libraries present on systems. Extremely useful for Attack Surface Management / Vulnerability Management. Returns a collated dataset of found software and versions, as well as a CSV list of systems found | Yes | Yes: Qualys subscription required, as well as CSAM subscription and agent-based scans on hosts |
RIPE WHOIS | Threat Intel | Look up IP address information: ownership, CIDR and geolocation information | No | No |
Shodan | Threat Intel | Query Shodan for IP address or host information, as well as performing count and search queries. Results will include the original Shodan JSON blob as a download |
Yes | No: basic functionality Yes: pagination, search queries, etc. |
SSLMate | Threat Intel | Look up SSL/TLS SHA256 hashes in the Certificate Transparency logs. Returns historic data, related hostnames, revocation status and validity times | Yes | No |
ThreatFox | Threat Intel | Query ThreatFox for MD5/SHA1/SHA256 hashes, IP addresses | No | No |
TLSGrab | Threat Intel | Connect to the given IP address + port, and attempt to retrieve the TLS certificate CNs. Note: this is an OPSEC risk, because the bot will actively attempt to connect to the host/port! | No | No |
Tria.ge | Threat Intel | Search the tria.ge sandbox project for IPv4, IPv6, domains, urls, hashes, a Tria.ge ID, etc. Responses may include the malware sample, if available | Yes | No: basic functionality Yes: additional information |
Tweetfeed | Threat Intel | Query the Tweetfeed API for the given IoC/tag | No | No |
Unprotect.it | Threat Intel | Search the Unprotect.it project for information on TTPs, code snippets and detection rules. Returns code snippets and detection rules as a download, if available | No | No |
URLhaus | Threat Intel | Look up reputation info on URLhaus for URLs and MD5 / SHA1 / SHA256 URL hashes | No | No |
VirusTotal | Threat Intel | Search VirusTotal for IP addresses, MD5/SHA1/SHA256 hashes, URLs and domains. Returned results will include maliciousness, TTP sets, malware family names, etc., if available | Yes | No: basic functionality Yes: paid VT features, throttling limit removal, etc. |
WikiJS | Information Retrieval | Search through WikiJS pages' contents for the given search terms. Returns links to the pages where the contents were found | Yes | Yes: currently requires a Microsoft Azure Search instance that indexes the WikiJS instance (Note: this is a WikiJS limitation!) |
New Matterbot modules can be created. A boilerplate example can be found in the commands
directory.
requirements.txt
).matterfeed.py
goes through the modules
directory and will run all detected modules every (by default) 5 minutes, outputting the results to the specified channels. Every module has its own custom configuration: you'll need to check the individual directories for more information. For example, the WikiJS module requires you to have a WikiJS instance with GraphQL API access, as well as a Microsoft Azure Search instance. You'll need to put the API key etc. in its configuration for it to work properly.
1) Copy config.defaults.yaml
to config.yaml
and edit the settings.
2) For every module you want to use, check the respective configuration in modules/.../
. If you do not want to use a feed
module, the easiest way to disable it is to move the directory somewhere else (or delete it), so it will not be detected on startup.
3) Start up the matterfeed.py
and watch the logfile for errors.
matterbot.py
goes through the commands
directory and will start listening in every specified channel for every specified bind (command). Every module has its own custom configuration: you'll need to check the individual directories for more information. For example, the ChatGPT module requires you to have an OpenAI account with API access, and you'll need to put the API key etc. in the configuration for it to work.
1) Copy config.defaults.yaml
to config.yaml
and edit the settings.
2) For every module you want to use, check the respective configuration in commands/.../
. You must make create your own settings.py
for every module in the commands/.../
directory you want to use! This is necessary so the bot can override the default configuration from defaults.py
.. If you do not want to use a module, the easiest way to disable it is to move the directory somewhere else (or delete it), so it will not be detected on startup.
3) Start up the matterbot.py
.
For matterfeed.py
, it is relatively simple to copy an existing module and alter it to your own needs. Make sure to update the pathlib
construct to reflect the right module and directory names.
matterbot.py
is a fully asynchronous setup, which has both advantages and limitations. The example
command is a good place to learn more and start developing your own command handler. Pay particular attention to the description in the commands/example/command.py
file for more information on how to get started and to avoid common pitfalls.
MatterBot would not be possible without the amazing work and/or generous help of others. If I have erroneously failed to list you here, please let me know! In alphabetical order, the people/organisations/companies I would particularly like to thank are:
Additional thanks to AlienVault, Censys, Shodan, Tweetfeed, VirusTotal for providing good API documentation, letting me easily write plugins for their services.