IOC Extractor

[kienmarkdo]

Analyze each collected message and add it to an IOC table in a SQLite3 database if an IOC is present.

IOCs include:

• URLs (URLs and domains), CVEs, exposed IPs, hashes (SHA256, SHA1, MD5)

Sqlite3 table diagram

ID	Message ID	Channel ID	User ID	IOC Value	IOC Type	Original Message	Translated Messaged
1	M1 ID	C1 ID	User1 ID	2.3.4.5	IPv4	"Hey guys, I have an RDP session on IP 2.3.4.5. Anyone interested?"	null
2	M2 ID	C2 ID	User2 ID	CVE-2024-21410	CVE	"Я обнаружил, что компания X не исправила CVE-2024-21410 на своих серверах."	"I have discovered that Company X's has not patched CVE-2024-21410 on their servers."
3	M7 ID	C1 ID	User1 ID	2.3.4.5	IPv4	"I want to attack 2.3.4.5 with CVE-2024-21410. Willing to pay."	null
4	M8 ID	C1 ID	User1 ID	CVE-2024-21410	CVE	"I want to attack 2.3.4.5 with CVE-2024-21410. Willing to pay."	null

Use Regex to identify these. Hashes are fixed in length.

Use case:

• As company Y, I know what my IPs are. I will search in the database for my IP 2.3.4.5 to see if my IP is present. I discover that it is, I can investigate further. "Is my IP leaked? How was it leaked? Why are people talking about my IP?"
• "Give me a list of all hashes that are being discussed, so that I can run it against my company's antivirus software or VirusTotal to see if I can detect it or not.

[kienmarkdo]

Domain regex: r"\b(?:a-zA-Z0-9?.)+[a-zA-Z]{2,}(?<!.)\b" Thinks "and.And" is a valid domain name.

IPv4 regex: r"\b(?:\d{1,3}.){3}\d{1,3}\b" Does not think "118.99.81.204118.99.81.204" or "Transparent212.119.97.198" contain IPv4 addresses.