search

bromiley / olaf

Office365 Log Analysis Framework
GNU General Public License v3.0
81 stars 14 forks source link

CSV Format for Python Parser #1

Open arnydo opened 6 years ago

arnydo commented 6 years ago

Thank you for sharing these tools and a great webinar!

I have exported the audit logs via Powershell to a CSV.

When importing the CSV into the Python script I get an error that the data is malformed. Is there a particular format that this needs to be in aside form what is exported straight from Powershell?

Line 4996 appears to be malformed - please check. Skipping for now.
Line 4997 appears to be malformed - please check. Skipping for now.
Line 4998 appears to be malformed - please check. Skipping for now.
Line 4999 appears to be malformed - please check. Skipping for now.
Line 5000 appears to be malformed - please check. Skipping for now.

Script to export data:

Search-UnifiedAuditLog -StartDate 2018-07-08 -EndDate 2018-07-18 -ResultSize 5000 | export-csv .\olaf\auditlog.csv -NoTypeInformation

Thanks again!

twofishbluefish commented 6 years ago

There's a lot of "fill-in-the-gaps" with this project. To answer your question, take a look at line 46 of the olaf.py script:

json_audit_data = json.loads(line[3])

Basically the script reads in each line of the csv file and operates on the 3rd index or 4th column, which is RecordType. It seems it should be operating on the 7th index or 8th column (Operations).

Changing the script to json.loads(line[7]) works for me. I got as far as trying to load the dashboard in Kibana, but none of visuals exist so all I get is a dashboard with missing visuals. I will have to go through the dashboard json file and try to recreate them one by one. #