Support for Mythic C2 - Githubissues

Support for Mythic C2 - https://github.com/its-a-feature/Mythic

I just starting following the conversations to support Covenant C2 (https://github.com/outflanknl/RedELK/issues/23) and immediately saw the same issues exist for Mythic (everything in a DB, no log files). I'd love to see Mythic support for RedELK, so can you all outline exactly what you'll need from Mythic to support log ingestion?

For Ghostwriter, we went the route of building Mythic Sync as a standalone tool that connects and listens for events to get the data where we want it, but isn't an ideal solution.

Is the list below (from Covenant Feature Request) the same items you'll need from Mythic/any other C2?

Filebeat config to read the log file of Covenant on the c2 server
Logstash rules that receive and filter the log lines
Cron scripts running on the c2 server to copy relevant files (screenshots, downloaded files, etc) from the Covenant directory to the /home/scponly directory
Modified rsync script on elkserver to copy files from /home/scponly on c2 server.
Review of field names in rtops- index to check if they are relevant for c2 in general, or (still) are too Cobalt Strike dedicated.
- https://raw.githubusercontent.com/outflanknl/RedELK/master/elkserver/templates/redelk_kibana_index_rtops.json
Update on documentation

CC @its_a_feature

Great to see this initiative. Would be great to achieve this.

The stated list is still valid for any type of C2. With the exception of the rtops name fields check. We've been very busy working to a v2 of RedELK that is C2 agnostic. Progress can be found in the maindev branch and the v2 beta1 release we put out just a few days ago.

But before the list above can be worked on, Mythic should be able to output some logs. Cobalt Strike actually is a great example of the detail of desired logging. It has very detailed logs on the red team operator's actions on the implants (primary focus) and logs on generic events (secondary focus).

Once there are logs, we can check on filebeat configs, logstash rules and all the others. But it starts with the logs :)

Let me know if there are any questions on the logging setup. Happy to help.

(tagging @its-a-feature with his correct handle :-) )

Mythic tracks a lot more data than cobalt strike and is a lot more descriptive about it too. It'll be pretty easy for me to push an update for it. I'm looking at an output like this:

{"timestamp":"10\/07\/2020 21:59:01","mythic_object":"payload_creation","message":{"id":16,"uuid":"ebf9db55-3434-48a1-9e87-8d8efa67fc52","tag":"apfell payload created by mythic_admin","operator":"mythic_admin","creation_time":"10\/07\/2020 21:59:01","payload_type":"apfell","pcallback":null,"operation":"Operation Chimera","wrapped_payload":null,"deleted":false,"build_container":"apfell","build_phase":"success","build_message":"Successfully built!","callback_alert":true,"auto_generated":false,"task":null,"file_id":{"id":25,"agent_file_id":"59ea2727-e4ba-4094-b0a2-21de7bd09b9a","total_chunks":1,"chunks_received":1,"chunk_size":0,"task":null,"cmd":null,"complete":true,"path":".\/app\/files\/ebf9db55-3434-48a1-9e87-8d8efa67fc52","full_remote_path":"","host":"","is_payload":true,"is_screenshot":false,"is_download_from_agent":false,"file_browser":null,"filename":"apfell.js","delete_after_fetch":false,"operation":"Operation Chimera","timestamp":"10\/07\/2020 21:59:01","deleted":false,"operator":"mythic_admin","md5":"37fbb73b273ed70f737be4feacff16f1","sha1":"5cf28f44bd1641ff99cc3bdb8d252b2d2b9368f3"}}}

{"timestamp":"10\/07\/2020 21:59:53","mythic_object":"file_upload","message":{"id":26,"agent_file_id":"add65787-ee25-434a-a3ea-c8ddd228c33f","total_chunks":1,"chunks_received":1,"chunk_size":0,"task":33,"cmd":"upload","complete":true,"path":".\/app\/files\/200b55bb-287c-4f0a-8236-f3b567401b5a","full_remote_path":"","host":"","is_payload":false,"is_screenshot":false,"is_download_from_agent":false,"file_browser":null,"filename":"atlas.exe","delete_after_fetch":false,"operation":"Operation Chimera","timestamp":"10\/07\/2020 21:59:53","deleted":false,"operator":"mythic_admin","md5":"56c9d7df81483f6e6ab1610f5357ce80","sha1":"0e8afc90838bd0d8338ac0a02042ce7b59611deb"}}

{"timestamp":"10\/07\/2020 21:59:53","mythic_object":"task","message":{"id":33,"agent_task_id":"5cfcb9fa-dfee-4aaf-8a40-bc3ca5a9034b","command":"upload","command_id":89,"status_timestamp_preprocessing":"10\/07\/2020 21:59:53","status_timestamp_submitted":"10\/07\/2020 21:59:53","status_timestamp_processing":null,"status_timestamp_processed":null,"timestamp":"10\/07\/2020 21:59:53","callback":12,"operation":"Operation Chimera","operator":"mythic_admin","status":"submitted","original_params":"
{\"assembly_id\":\"atlas.exe\",\"remote_path\":\"atlas2.exe\"}","comment":"","comment_operator":null,"completed":false}}

{"timestamp":"10\/07\/2020 22:00:20","mythic_object":"response","message":{"task_id":"5cfcb9fa-dfee-4aaf-8a40-bc3ca5a9034b","user_output":"File successfully uploaded","status":"","completed":"true","total_chunks":"","full_path":"","chunk_num":"","chunk_data":"","file_id":""}}

{"timestamp":"10\/07\/2020 22:00:20","mythic_object":"task_completed","message":{"id":33,"agent_task_id":"5cfcb9fa-dfee-4aaf-8a40-bc3ca5a9034b","command":"upload","command_id":89,"status_timestamp_preprocessing":"10\/07\/2020 21:59:53","status_timestamp_submitted":"10\/07\/2020 21:59:53","status_timestamp_processing":"10\/07\/2020 21:59:59","status_timestamp_processed":null,"timestamp":"10\/07\/2020 21:59:59","callback":12,"operation":"Operation Chimera","operator":"mythic_admin","status":"processing","original_params":"{\"assembly_id\":\"atlas.exe\",\"remote_path\":\"atlas2.exe\"}","comment":"","comment_operator":null,"completed":true}}

extra newlines added by me for clarity. Essentially, they'd all be JSON entries (one per line) with a timestamp (UTC), a mythic_object field to give you some context about what you're looking at, and then the message field has the actual JSON message from mythic. Thoughts?

This capability is included in Mythic 2.1.1. Specifically, I initially only included the following types:

file_upload (file staged on mythic as part of tasking with the intent to get sent to the agent)
file_manual_upload (file staged on mythic as part of a user manually hosting it)
file_screenshot (file is a screenshot from the agent)
file_download (file is downloaded from agent to mythic)
artifact_new (new artifact created - think IOC)
eventlog_new (new eventlog message)
eventlog_modified (eventlog was modified, like resolving an issue or changing their message)
payload_new (new payload created)
task_mitre_attack (a task was associated with a new mitre attack technique)
task_new (a new task was created)
task_completed (a task completed)
task_comment (somebody added/removed/edited a comment on a task)
credential_new (a new credential was added to the store)
credential_modified (a credential was modified)
response_new (a new response for the user to see)
keylog_new (a new keylog entry)
callback_new (new callback registered)

I can add more things, just figured that was a decent start. To enable the feature within Mythic, add the desired log name to Mythic/mythic-docker/config.json in the siem_log_name field. If it's empty, no log. If there's a value, that file will be created and the logs will be written there.

There's a lot of stuff to parse, so I'm hoping that these values (mythic_object) will make it easier to parse out what you're looking at for SIEM ingestion.

Great to see a C2 creator that has already thought about logging 👍

JSON log format should be workable. Havent used it yet, but Elastic has support for incoming JSONs. I expect no issues there.

Looking at your example logs I see a lot of data that can be mapped to the fields that we track for RT operations. Could you have a look at the FieldNames list of rtops as well (https://github.com/outflanknl/RedELK/blob/maindev/example-data-and-configs/RedELKFieldnamesV2.md#Index-rtops) and let me know if there are data points missing within Mythic? I assume not, but just to be sure.

Another aspect besides the logs are the actual screenshots and downloaded files. I assume these are also stored on the c2 server. Typically we would like to also transfer these over to the RedELK server. ao for easy accessing from the Kibana interface. With Cobalt Strike we take the approach of periodic rsyncing the files from C2 back to RedELK server, and presenting them in the /var/www folder, where Kibana has clickable links to the specific files. Would this approach fit Mythic as well?

Glancing through it, I think pretty much everything is covered. Since Mythic has everything broken out into individual Docker containers, the C2 profile information might need to be pulled from the C2 container itself vs via a specific log that Mythic generates. This is because all the C2 profiles govern how their own stuff works within their Docker containers, so the HTTP profile for example is pretty straight forward with static headers, port, etc, but the dynamicHTTP profile (think CS' malleable c2) is a JSON configuration that's a lot more expressive, so we'd just have to think about how we'd want to handle it.

For the files, Mythic stores all files in Mythic/mythic-docker/app/files. So, it's easy enough to use rsync or something on that folder to RedELK; however, to help with potential pathing vulnerabilities, Mythic stores all files with a filename as a random UUID string. The real filename is stored in a database field associated with a File along with the real path on disk (i.e. this file data entry has a real name of "../../../malicious.evil" and is stored on disk as this random UUID). So, just shuttling off the files won't mean too much to something looking at them in the UI, but since you're also getting all of the file data, you can do the tracking. As an example:

{"id":25,"agent_file_id":"59ea2727-e4ba-4094-b0a2-21de7bd09b9a","total_chunks":1,"chunks_received":1,"chunk_size":0,"task":null,"cmd":null,"complete":true,"path":".\/app\/files\/ebf9db55-3434-48a1-9e87-8d8efa67fc52","full_remote_path":"","host":"","is_payload":true,"is_screenshot":false,"is_download_from_agent":false,"file_browser":null,"filename":"apfell.js","delete_after_fetch":false,"operation":"Operation Chimera","timestamp":"10\/07\/2020 21:59:01","deleted":false,"operator":"mythic_admin","md5":"37fbb73b273ed70f737be4feacff16f1","sha1":"5cf28f44bd1641ff99cc3bdb8d252b2d2b9368f3"}

^ notice the path is the actual random UUID and the filename is the real filename.

@its-a-feature that should be enough to start with, looking quickly at the sample log you posted it looks like we'll have plenty of information to put in 👍 Would you maybe have a sample log file containing all kind of outputs we could use to create the logstash parser?

Awesome! Yup, here's examples:

{"timestamp":"10\/09\/2020 17:01:28","mythic_object":"task_new","message":{"id":39,"agent_task_id":"269ce9b4-b949-4f6d-b193-747f19b694c3","command":"screenshot","command_id":84,"status_timestamp_preprocessing":"10\/09\/2020 17:01:28","status_timestamp_submitted":"10\/09\/2020 17:01:28","status_timestamp_processing":null,"status_timestamp_processed":null,"timestamp":"10\/09\/2020 17:01:28","callback":13,"operation":"Operation Chimera","operator":"mythic_admin","status":"submitted","original_params":"","comment":"","comment_operator":null,"completed":false}}
{"timestamp":"10\/09\/2020 17:01:28","mythic_object":"task_mitre_attack","message":{"id":18,"attack":"T1113","attack_name":"Screen Capture","task":39,"task_command":"screenshot","task_params":""}}
{"timestamp":"10\/09\/2020 17:01:38","mythic_object":"file_screenshot","message":{"id":39,"agent_task_id":"269ce9b4-b949-4f6d-b193-747f19b694c3","command":"screenshot","command_id":84,"status_timestamp_preprocessing":"10\/09\/2020 17:01:28","status_timestamp_submitted":"10\/09\/2020 17:01:28","status_timestamp_processing":"10\/09\/2020 17:01:38","status_timestamp_processed":null,"timestamp":"10\/09\/2020 17:01:38","callback":13,"operation":"Operation Chimera","operator":"mythic_admin","status":"processing","original_params":"","comment":"","comment_operator":null,"completed":false}}
{"timestamp":"10\/09\/2020 17:01:50","mythic_object":"task_completed","message":{"id":39,"agent_task_id":"269ce9b4-b949-4f6d-b193-747f19b694c3","command":"screenshot","command_id":84,"status_timestamp_preprocessing":"10\/09\/2020 17:01:28","status_timestamp_submitted":"10\/09\/2020 17:01:28","status_timestamp_processing":"10\/09\/2020 17:01:38","status_timestamp_processed":null,"timestamp":"10\/09\/2020 17:01:38","callback":13,"operation":"Operation Chimera","operator":"mythic_admin","status":"processing","original_params":"","comment":"","comment_operator":null,"completed":true}}
{"timestamp":"10\/09\/2020 17:01:50","mythic_object":"response_new","message":{"id":27,"response":"{\"file_id\":\"a4fcf1d8-9039-4cff-bbdf-85069dc64c4c\"}","timestamp":"10\/09\/2020 17:01:50","task":{"id":39,"agent_task_id":"269ce9b4-b949-4f6d-b193-747f19b694c3","command":"screenshot","command_id":84,"status_timestamp_preprocessing":"10\/09\/2020 17:01:28","status_timestamp_submitted":"10\/09\/2020 17:01:28","status_timestamp_processing":"10\/09\/2020 17:01:38","status_timestamp_processed":null,"timestamp":"10\/09\/2020 17:01:38","callback":13,"operation":"Operation Chimera","operator":"mythic_admin","status":"processing","original_params":"","comment":"","comment_operator":null,"completed":true}}}
{"timestamp":"10\/09\/2020 17:03:24","mythic_object":"artifact_new","message":{"id":2,"task_id":40,"task":"whoami","command":"shell","timestamp":"10\/09\/2020 17:03:24","artifact_instance":"sh -c whoami","artifact_template":"Process Create","operation":"Operation Chimera","host":"itss-mac.local"}}
{"timestamp":"10\/09\/2020 17:03:29","mythic_object":"task_completed","message":{"id":40,"agent_task_id":"8cb3296d-26f0-4590-9ff1-d38d39e46cde","command":"shell","command_id":44,"status_timestamp_preprocessing":"10\/09\/2020 17:03:24","status_timestamp_submitted":"10\/09\/2020 17:03:24","status_timestamp_processing":"10\/09\/2020 17:03:29","status_timestamp_processed":null,"timestamp":"10\/09\/2020 17:03:29","callback":13,"operation":"Operation Chimera","operator":"mythic_admin","status":"processing","original_params":"whoami","comment":"","comment_operator":null,"completed":true}}
{"timestamp":"10\/09\/2020 17:03:50","mythic_object":"file_upload","message":{"id":33,"agent_file_id":"6dfd02e8-bbbe-4b1f-9404-522ee726fd12","total_chunks":1,"chunks_received":1,"chunk_size":0,"task":41,"cmd":"upload","complete":true,"path":".\/app\/files\/27a03595-3bd3-422c-9006-51876cfc60b0","full_remote_path":"","host":"","is_payload":false,"is_screenshot":false,"is_download_from_agent":false,"file_browser":null,"filename":"bug-feature.png","delete_after_fetch":false,"operation":"Operation Chimera","timestamp":"10\/09\/2020 17:03:50","deleted":false,"operator":"mythic_admin","md5":"d2f2b98236039af1a0f3364f8bec7df6","sha1":"e630c1805f47f4ede9cf8a396a8bad1327254f2e"}}
{"timestamp":"10\/09\/2020 17:04:33","mythic_object":"payload_new","message":{"id":18,"uuid":"572cc106-e9ba-45a2-8eac-71fff0483969","tag":"poseidon payload created by mythic_admin","operator":"mythic_admin","creation_time":"10\/09\/2020 17:04:20","payload_type":"poseidon","pcallback":null,"operation":"Operation Chimera","wrapped_payload":null,"deleted":false,"build_container":"poseidon","build_phase":"success","build_message":"Created payload!\n","callback_alert":true,"auto_generated":false,"task":null,"file_id":{"id":34,"agent_file_id":"9a34e932-87fb-4c68-87bd-a9476c609e6d","total_chunks":1,"chunks_received":1,"chunk_size":0,"task":null,"cmd":null,"complete":true,"path":".\/app\/files\/572cc106-e9ba-45a2-8eac-71fff0483969","full_remote_path":"","host":"","is_payload":true,"is_screenshot":false,"is_download_from_agent":false,"file_browser":null,"filename":"poseidon.bin","delete_after_fetch":false,"operation":"Operation Chimera","timestamp":"10\/09\/2020 17:04:20","deleted":false,"operator":"mythic_admin","md5":"95da327842a3ec2bcfe42661f174579a","sha1":"b9d20300c478eb60f8091dda026b14ec284c3688"}}}
{"timestamp":"10\/09\/2020 17:04:51","mythic_object":"task_comment","message":{"id":39,"agent_task_id":"269ce9b4-b949-4f6d-b193-747f19b694c3","command":"screenshot","command_id":84,"status_timestamp_preprocessing":"10\/09\/2020 17:01:28","status_timestamp_submitted":"10\/09\/2020 17:01:28","status_timestamp_processing":"10\/09\/2020 17:01:38","status_timestamp_processed":null,"timestamp":"10\/09\/2020 17:01:50","callback":13,"operation":"Operation Chimera","operator":"mythic_admin","status":"processing","original_params":"","comment":"almost caused a dialog popup for the user! Yikes!","comment_operator":"mythic_admin","completed":true}}
{"timestamp":"10\/09\/2020 17:05:05","mythic_object":"eventlog_new","message":{"id":167,"operator":"mythic_admin","timestamp":"10\/09\/2020 17:05:05","message":"did we get caught?!","operation":"Operation Chimera","level":"warning","deleted":false,"resolved":false}}
{"timestamp":"10\/09\/2020 17:05:06","mythic_object":"eventlog_modified","message":{"id":167,"operator":"mythic_admin","timestamp":"10\/09\/2020 17:05:05","message":"did we get caught?!","operation":"Operation Chimera","level":"warning","deleted":false,"resolved":true}}
{"timestamp":"10\/09\/2020 17:05:44","mythic_object":"credential_new","message":{"id":4,"type":"plaintext","task":null,"task_command":null,"account":"itsafeature","realm":"itss-mac.local","operation":"Operation Chimera","timestamp":"10\/09\/2020 17:05:44","credential":"SuperSecurePasswordHere!!#@","operator":"mythic_admin","comment":"pulled from the screenshot yo","deleted":false}}
{"timestamp":"10\/09\/2020 17:05:50","mythic_object":"credential_modified","message":{"id":4,"type":"plaintext","task":null,"task_command":null,"account":"itsafeature","realm":"itss-mac.local","operation":"Operation Chimera","timestamp":"10\/09\/2020 17:05:44","credential":"SuperSecurePasswordHere!!#@3","operator":"mythic_admin","comment":"pulled from the screenshot yo","deleted":false}}
{"timestamp":"10\/09\/2020 17:06:14","mythic_object":"callback_new","message":{"id":14,"agent_callback_id":"01e03e36-ee90-4bfd-a316-f725baf3dd84","init_callback":"10\/09\/2020 17:06:14","last_checkin":"10\/09\/2020 17:06:14","user":"itsafeature","host":"itss-mac.local","pid":4211,"ip":"192.168.205.130","external_ip":null,"description":"apfell payload created by mythic_admin","operator":"mythic_admin","active":true,"pcallback":null,"registered_payload":"34cd7b0a-1e10-424f-8627-4e5ea2ba9446","payload_type":"apfell","payload_type_id":3,"payload_description":"apfell payload created by mythic_admin","integrity_level":2,"locked":false,"locked_operator":null,"operation":"Operation Chimera","os":"macOS Version 10.15.6 (Build 19G2021)","architecture":"x64","domain":null,"port":null,"socks_task":null,"extra_info":"","sleep_info":""}}
{"timestamp":"10\/09\/2020 17:06:36","mythic_object":"file_manual_upload","message":{"id":35,"agent_file_id":"08f37889-266d-44b8-a868-85e404538c86","total_chunks":1,"chunks_received":1,"chunk_size":0,"task":null,"cmd":null,"complete":true,"path":".\/app\/files\/08f37889-266d-44b8-a868-85e404538c86","full_remote_path":"","host":"","is_payload":false,"is_screenshot":false,"is_download_from_agent":false,"file_browser":null,"filename":"HealthInspector.js","delete_after_fetch":false,"operation":"Operation Chimera","timestamp":"10\/09\/2020 17:06:36","deleted":false,"operator":"mythic_admin","md5":"664e75e9146c524ac2823f502d917e68","sha1":"0b9630d96db69d3f0c6f5a6393d9267c8eed1892"}}

Is RedELK updated with the latest MITRE ATT&CK Sub-Techniques? That's one piece that I still need to update. I'm hoping it won't change the messages too much, but I haven't looked into it yet from a programming/logging perspective.

Thanks for the example data! Really helps us getting started (when we find the time).

Wrt ATT&CK nrs, there are two parts:

RedELK just stores in the Elasticsearch fields for ATT&CK whatever we think there should be. It doesn't differentiate between old and new style. The dashboards (e.g. https://twitter.com/MarcOverIP/status/1311991775612014593/photo/2) just show whatever there is in the relevant fields.
The integrated MITRE ATT&CK viewer (e.g. https://twitter.com/MarcOverIP/status/1311991775612014593/photo/1) is still tuned to old style. Main reason for this is that CS (and our inhouse C2) are still working with the old style. Its an easy change to do.

So if Mythic still does old style, you are on par with our slow dev speed :-D But even if you would get ahead of us, RedELK would be ready for parsing new style.

Has there been any progress on this? Was curious in using this integration for a red vs blue exercise.

AFAIk there hasn't been any movement on this side, but Mythic 3.0 now has logging as a separate docker container (ex: https://github.com/MythicC2Profiles/basic_logger), so you have full control over logging formats and what to do with the data. The container just listens for logging events over RabbitMQ and by default just writes them to stdout and optionally a file. Since you control that docker container though, you can update it to change formats and submit to RedELK in any format (or any SIEM). It would be cool if there was a supported RedELK logging container that did the format adjustments (if any are needed) for you

Apol for the extreme delay by us on this. We have not made progress on this. Just many other things in life and work that require our attention. We appreciate any input from others on this though!

outflanknl / RedELK

Support for Mythic C2 #63