Sigma.csv Formatting

[b1draper]

In version 2.0.0-beta.2 - There appears to be a formatting issue with the sigma.csv file. In the "Event Data" column the data appears as blank or as "---" due to formatting and a few carriage returns. When processing the same data with the older version 1.1.7 the formatting issue didn't show up since I believe that the "sigma.csv" might be a new report

Example Showing how it displays in Timeline Explorer: 

Example showing copy of sanitized data within one of the rows in that column: 

[alexkornitzer]

Hi @b1draper,

They come in for me but with the sigma-event-logs-all mapping the EventData is stored as a YAML string. Thus my assumption here is that the tool you are using in the above has not expanded the cell to show you all of its contents, or it is not handling carriage returns within strings correctly.

What tool are you using, then I could have a look, or maybe we need to change the EventData output format for CSV?

[b1draper]

Hello Alex,

Thanks for getting back to me so quickly. I'm viewing the CSV file using Eric Zimmerman's Timeline Explorer. I imagine that if I viewed the files within Excel it would probably be ok. But Timeline Explorer is a DFIR mainstay and handles large files really well. The syntax I'm using is as follows. ./chainsaw hunt /mnt/h/Customer/FS1/export/eventlogs/ --mapping ./mappings/sigma-event-logs-all.yml --sigma ./sigma/rules/ -r ./rules/ --csv --output /mnt/h/customer/FS1/export/newchainsawtest2/

Thanks for the help,

Brian

On Fri, Jul 22, 2022 at 9:52 AM Alex Kornitzer @.***> wrote:

Hi @b1draper https://github.com/b1draper,

They come in for me but with the sigma-event-logs-all mapping the EventData is stored as a YAML string. Thus my assumption here is that the tool you are using in the above has not expanded the cell to show you all of its contents, or it is not handling carriage returns within strings correctly.

[image: Screenshot 2022-07-22 at 14 48 45] https://user-images.githubusercontent.com/2750747/180453440-9d2d6285-abfe-4c29-9a2e-fd581daf9e18.png

What tool are you using, then I could have a look, or maybe we need to change the EventData output format for CSV?

— Reply to this email directly, view it on GitHub https://github.com/countercept/chainsaw/issues/89#issuecomment-1192599037, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKTCECEKBB35BLMEPIXKPG3VVKRR7ANCNFSM54LGI4UQ . You are receiving this because you were mentioned.Message ID: @.***>

[alexkornitzer]

No worries,

Right so I just tried it and the data is there, if you hover or double click on the cell it will show the full contents. I can't see a way to expand the cell height, so it might be worth putting that in as a feature request to Timeline Explorer?

I am not sure if its worth changing Chainsaw side as the data is more readable in this format, but am very open to suggestions. We could potentially flatten out the YAML.

[b1draper]

Thanks for the help.

I agree that the data looks very readable and is formatted nicely. However, would it be possible to remove the carriage returns and replace them with additional spaces that way they're separated but still on the same line allowing Timeline explorer to structure the data that's displayed. I'll also contact Eric and mention the problem. Since he just added support to run chainsaw from within KAPE other people are likely to experience the same issues.

Thanks agian

On Fri, Jul 22, 2022 at 11:07 AM Alex Kornitzer @.***> wrote:

No worries,

Right so I just tried it and the data is there, if you hover or double click on the cell it will show the full contents. I can't see a way to expand the cell height, so it might be worth putting that in as a feature request to Timeline Explorer?

I am not sure if its worth changing Chainsaw side as the data is more readable in this format, but am very open to suggestions. We could potentially flatten out the YAML.

— Reply to this email directly, view it on GitHub https://github.com/countercept/chainsaw/issues/89#issuecomment-1192668774, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKTCECBHO4LXRDSWNBS2G7DVVK2JTANCNFSM54LGI4UQ . You are receiving this because you were mentioned.Message ID: @.***>

[EricZimmerman]

it already does that. if there are CRLFs and whatnot, the rows expand. to see this, dump an MFT to csv and load it, then look in the Zone id column and you will see how the data shows up.

there does not seem to be anything to do here. if there are CRLFs and the CSV is properly formatted, TLE will expand the rows.

[alexkornitzer]

@EricZimmerman, thanks for the clarification and example, super helpful. In that case I have a feeling then it is because Chainsaw is only using \n and not doing CRLF. I will dig into it now.

[alexkornitzer]

Hi @EricZimmerman,

I must be doing something incredibly wrong because even with a very basic CSV I cannot get Timeline Explorer (1.3 or 2.0) to expand the row. I can tell it has parsed it correctly because the tooltip and the cell view both recognise the line break and format it correctly. Do you have any thoughts as to what I could be doing wrong here?

[EricZimmerman]

What does it do in 2.0?

I don't see why it would not work

[alexkornitzer]

2.0 is displaying it in exactly the same way as 1.3 for me.

[alexkornitzer]

Okay I have worked it out, Timeline Explorer is only enabling multiline support if Zone is in the column name hence the cells not being expanded. I am going to close this issue out as it is not a Chainsaw bug.

[EricZimmerman]

Ah okay so in that case simply creating a plugin to support chainsaw for timeline explorer would solve this issue on all fronts and any other format

[b1draper]

Guys,

Thanks for working on this.

On Sat, Jul 23, 2022 at 9:46 AM Eric @.***> wrote:

Ah okay so in that case simply creating a plugin to support chainsaw for timeline explorer would solve this issue on all fronts and any other format

— Reply to this email directly, view it on GitHub https://github.com/countercept/chainsaw/issues/89#issuecomment-1193128384, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKTCECBX6CGNNQEQJS7WXF3VVPZUDANCNFSM54LGI4UQ . You are receiving this because you were mentioned.Message ID: @.***>

[FranticTyping]

I would maybe suggest making a toggle option for Timeline Explorer to expand the column as opposed to a plugin specifically for Chainsaw? Otherwise there would likely need to be a plugin for each CSV format moving forwards, as opposed to one generic solution.

Either way, it sounds like we have identified that the support needs to be added on the TimelineExplorer side for this issue. Thanks for everyone's work figuring this out! 👍

[alexkornitzer]

Additionally @b1draper if you just wanna view the data now and not wait for changes if you add Zone into the name for Event Data field this will trick TE into using multiline.

[EricZimmerman]

That suggestion won't work because I would have to make every single field a particular type of editor

It's trivial to make a new plugin for CSV formats that are going to be used a lot which is the right approach anyway because it gives you strongly type data versus treating everything as strings

I could certainly add another line like currently exists where it looks for the string multi and enables such a feature which then makes it dynamic based on the column header

[b1draper]

Good Morning Eric,

The work around that Alex mentioned works but (but it is not very easy for me to articulate to other analysts that I know). I was curious if you're still planning on writing the plug-in for the chainsaw sigma.csv file?

Thanks for your help

On Sat, Jul 23, 2022 at 10:07 AM Eric @.***> wrote:

That suggestion won't work because I would have to make every single field a particular type of editor

It's trivial to make a new plugin for CSV formats that are going to be used a lot which is the right approach anyway because it gives you strongly type data versus treating everything as strings

I could certainly add another line like currently exists where it looks for the string multi and enables such a feature which then makes it dynamic based on the column header

— Reply to this email directly, view it on GitHub https://github.com/countercept/chainsaw/issues/89#issuecomment-1193131280, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKTCECCHB3QCNKNYPWZO6KTVVP4C5ANCNFSM54LGI4UQ . You are receiving this because you were mentioned.Message ID: @.***>

[EricZimmerman]

i wasnt planning on it, but the plugins are open source. anyone can write it. you may be able to convince andrew to write a plugin for it, but i would have to tweak things in TLE proper to detect the column that has multiple lines.

its a tricky problem to balance. the plugin is easy, the second part is more difficult