[RFF] JSON output logging

[aress31]

Right now parsing result outputs from CME is not really intuitive and requires a lot of crafty bash one-liners.

To address this issue and improve on the tool, how about offering JSON output (with --json) to allow for easily parsing (e.g. extract hosts, users, passwords, shares) with the help of jq and ConvertFrom-JSON.

[sandervandegeijn]

I really really would this option as well. It does exactly what we need, but I need to enrich the data with our CMDB. To do that I need to parse the data somehow. JSON output would be much appreciated.

[SmashRocket]

Would be a great feature +1

[Senne-s]

Is there any update on this? I would like to be able to properly parse the output data from the CME scripts but there isn't really a way to efficiently do this.

[mpgn]

The sponsors version has a log feature where everything is send to a file.

[NeffIsBack]

Alternately you can export the data from cmedb as csv if the usual log output doesn't suite you

[daanwa]

I'm also interested in JSON output.

[NeffIsBack]

@Marshall-Hallenbeck maybe a feature for the new cmedb? Is there an easy way to export as json?

[Marshall-Hallenbeck]

@aress31 @ict-one-nl @SmashRocket @Senne-s @daanwa Do you mean the entire output (so instead of the normal output, it's all JSON), or just an option to have normal STDOUT prints and JSON to a file, or a way to dump CMEDB information as JSON? Could you give some examples and I can see what I can put together to fit the needs.

Use cases on how you're interacting with results, and how it would look with JSON would be appreciated.

[daanwa]

I want the results of the tool to be incorporated into our tooling. However, the current output shown on STDOUT is somewhat cumbersome to parse. So in my case it would be most handy to have a command line flag that outputs the results in JSON to a file.

Preferably I would not have to use the cmedb cli and I could just provide the JSON output flag to the cme app.

[Marshall-Hallenbeck]

@daanwa Thanks for that information. It might be pretty hard to do, but I can look into it. Can you provide an example of how you would expect the JSON structure to look like with several hosts & modules?

[sandervandegeijn]

Yup, I'd like to parse the output with something like a python script to write to opensearch and enrich the info with contents of our CMDB :)

[mpgn]

i'm not sure to see the link between crackmapexec and opensearch

[sandervandegeijn]

That's up to my scripting skills (to ingest data into our SIEM and generate reports). But to do anything useful I need to interact with the data, for that I'd need parsable output; json works fine for that :)

[Marshall-Hallenbeck]

@ict-one-nl What sort of structure would you expect as output? Can you provide an example?

[sandervandegeijn]

Hmm, doesn't really matter, I can image a header like object for the general stuff and a list of objects for each line outputted by CME containing all the details for that line?

[daanwa]

Agree with @ict-one-nl, doesn't matter a whole lot as long as it is consistent. But I can imagine something such as the JSON below (depending on which command you run of course):

{
  "smb": [
    {
      "host": "hostname or ip",
      "user": "username",
      "password": "password",
      "domain": "domain",
      "status": "STATUS_LOGON_FAILURE"
    }
  ]
}

Potentially with some additional keys that make it easier to determine if the command is considered successful, i.e. when we receive a STATUS_LOGON_FAILURE there is an extra key of "login": false or something.

[mpgn]

but what's the point ? I don't understand

[sandervandegeijn]

Why you would want a parseable format? Just like nmap output; I can use it to create reports, cross reference with the CMDB, do lookups for owners of systems in our databases, write a text file with HEY THIS SHARE IS OPEN!!, make it searchable in a tool like elastic, put in our SIEM, etc. Possibilities are endless. This is not only about targeting and exploiting one host, it's also about doing recon, integrating with other systems and providing an overview. It makes finding vulnerable hosts much much easier.

[daanwa]

Exactly. It is mainly aimed at integrating with some of our other tooling or for collecting additional authentication information about certain data points.

@mpgn Perhaps there is already a different way that can do this and I'm just not aware of it? Currently I'm parsing the sqlite databases but I've seen that not all commands/modules populate the db's.

[sandervandegeijn]

Using the SQLite db quite cumbersome for what's needed in this case. Just a different formatter for the data that's already there would suffice :)

To give some more context, I'm working at a university in the Netherlands on the blue team. Example use case: I want to use this to scan our ±16k hosts to find vulnerable stuff and open shares that contain data that shouldn't be there. I'll probably combine masscan with CME to go through the network quickly. It's also about chaining tools :) Extensibility is nice :)

[sandervandegeijn]

Okay, did it the sqlite way for now:

sqlite3 -json /root/.cme/default/smb.db 'select * from shares inner join computers on computers.hostname = shares.computerid where read = true and name NOT LIKE "%print%"' > /root/.cme/output.json

[mpgn]

So, my 2 cents, if we add a json option it should be used in combinaison with --log file option.

cme <proto> <ip> -u <user> -p <pass> --log filename --json

And the output in the log file 'filename' will be in json. I know @NeffIsBack already work on this, I think this is the best way to do it.

[daanwa]

Yeah that would work for me!