search

trellix-enterprise / mysql-audit

AUDIT Plugin for MySQL. See wiki and readme for description. If you find the plugin useful, please star us on GitHub. We love stars and it's a great way to show your feedback.
Other
238 stars 57 forks source link

Option to to log directly to a syslog server. #14

Open srinathman opened 12 years ago

srinathman commented 12 years ago

It would be nice to be able to send logs directly to a syslog server.

glicht commented 12 years ago

You can achieve this by using syslog-ng and reading incoming log messages from a socket with the no-parse option.

For example:

Configure syslog-ng to listen on a Unix socket and send the incoming messages to a log file:

source s_myaudit { unix-stream("/var/lib/mysql/mysql-audit.sock" flags(no-parse)); };
destination d_myaudit { file("/var/log/mysql-audit"); };
log { source(s_myaudit); destination(d_myaudit); };

Then configure the audit plugin to log to a Unix socket by setting:

audit_json_socket_name=/var/lib/mysql/mysql-audit.sock
audit_json_socket=ON

Message will go through syslog-ng and arrive at the file /var/log/mysql-audit.

In the same way you can change syslog-ng to send the messages to an external syslog deamon over the network. For example:

source s_myaudit { unix-stream("/var/lib/mysql/mysql-audit.sock" flags(no-parse)); };
destination d_mynet { tcp("XXX.XXX.XXX.XXX" port(514); };
log { source(s_myaudit); destination(d_mynet); };
srinathman commented 12 years ago

Thanks, do you know if it's possible to do the same using rsyslog ?

glicht commented 12 years ago

Not too familiar with rsyslog so can't say if it possible or not. I would expect it to be possible as this is fairly basic.

Guy On Oct 12, 2012 2:25 PM, "srinathman" notifications@github.com wrote:

Thanks, do you know if it's possible to do the same using rsyslog ?

— Reply to this email directly or view it on GitHubhttps://github.com/mcafee/mysql-audit/issues/14#issuecomment-9374827.

bmurphy96 commented 11 years ago

I just spent three hours upgrading syslog on a system to rsyslog, tweaking the config, creating a socket and pointing the audit-plugin to it. The plugin fails with the following message:

Audit Plugin: unable to connect to socket: /tmp/audit.sock. err: Protocol wrong type for socket. audit socket handler disabled!!

The is because rsyslog uses the datagram based socket file while (I assume) the audit plugin is expecting a stream-based socket. By default syslog-ng uses stream-based sockets (this is configurable). Unfortunately it appears that rsyslog does not have the ability to change from datagram to stream based.

http://lists.adiscon.net/pipermail/rsyslog/2011-February/027988.html

So it appears that you have to use syslog-ng to use the audit plugin with sockets. Hope that helps someone.

glicht commented 11 years ago

You are correct. The AUDIT plugin is stream based. I guess this is a point for improvement, to support also other syslog daemons by using datagram based sockets.

Thanks for the update.

srinathman commented 11 years ago

Finally I migrated to syslog-ng.

halides commented 11 years ago

Wondering if there is any work done on the datagram sockets? I've been taking a look at the audit codebase - as I am quite unfamiliar with C++ I'd be happier with a few more comments in the source code :-) I'm willing to work on adding the DGRAM sockets, but I'd like a bit of mentoring here... Anyone on IRC@freenode willing to give me a few pointers?

glicht commented 11 years ago

Hi,

No work has been done yet on this issue.

I can help you out. I am not on IRC but you can post questions freely on this thread.

In general the change isn't big. You need to change Audit_socket_handler to also support DGRAM sockets. I would go with adding support for DGRAM by checking if the socket name starts with the prefix: "dgram:". For example: 

dgram:/tmp/json.sock

You will get this value from setting the option audit_json_socket_name and it is saved at the member: Audit_socket_handler::m_sockname. Functions I see you will need to modify:

halides commented 11 years ago

Thanks, I've been digging around Audit_socket_handler quite a lot already; off for today - I'll get back to you tomorrow!

halides commented 11 years ago

Sheesh! That was a long "tomorrow", eh?

My xubuntu updated MySQL to 5.5.29, took a while until I figured out how to extract the offsets (couldn't do that against the ubuntu packed binary) and the correct place to put them (32 vs. 64, heh). I now can even uninstall the plugin, this crashes the mysqld binary though! I'm doing the uninstall rather crudely (a quick hack) in the code though, might be my changes in there... Anyhow, wanted to let you know I'm still on the case!

.p

EDIT:

I took a closer look at the segfault problem. What I did was simply comment out the if (!uninstall_plugin_enable) block on line 917-922 so every call to uninstall it should succeed. This however segfaults the server. From what I read and understand, commenting said block out should not cause such behavior. Any thoughts?

glicht commented 11 years ago

Hi Halides,

I suggest opening a new issue for the crash as it is not related to syslog. In the new issue please post the output seen in the mysql error log after the crash (hopefully it should contain a stack trace).

halides commented 11 years ago

Hi, just an update, created a socket handler for datagram also, quite straightforward. Writer has to wait until next week, though!

glicht commented 11 years ago

Thanks for the update.

Sent from my mobile device On Jan 30, 2013 2:15 PM, "halides" notifications@github.com wrote:

Hi, just an update, created a socket handler for datagram also, quite straightforward. Writer has to wait until next week, though!

— Reply to this email directly or view it on GitHubhttps://github.com/mcafee/mysql-audit/issues/14#issuecomment-12888860.

halides commented 11 years ago

Communicating with rsyslogd through a datagram socket is working, in the end it was really simple to get it going. I just tried a bit of this and that and by some wonder it started working without me really deeply knowing what's going on :-)

Some criticism: the codebase is messy and uncommented - thus hard to read and understand. Adding to that my complete inexperience with the MySQL plugin system it took a lot longer time than it should to get the hang of things and I'm still quite baffled on how the system does what it does. I'll dive in deeper in the coming days, though.

glicht commented 11 years ago

Thanks for the update. When ready feel free to post a pull request with your changes.

Regarding the code base, I agree it needs some cleanup and better documentation. Not sure if there will be any changes soon, so just feel free to post any questions you have here.

halides commented 11 years ago

Hi, sorry for the sluggish updates, been quite ill for a long time.

I fetched the current version and decided to start from the beginning and merge changes by hand - I did some refactoring of the codebase to make more sense of it for me and merging now would be a big mess. Are you open to receiving a refactored and re-styled codebase or would you rather I give you just the patch and you can see when you have time to clean it up yourself? Quick stuff I could clean up which ran into: a bunch of unused and/or redundant variables hanging around we could get rid of - the code is at times intended badly - bracing and commenting waste space which makes the codebase hard to read.

What do you think?

glicht commented 11 years ago

Hi,

Hope you are feeling well.

I think it is best to limit the refactoring and re-styling changes as it makes it hard to merge the changes. You can submit the patch as a pull request and then we can start reviewing it and see how complicated it is to get it in as part of the code base.

ruckc commented 10 years ago

+1, i'm generating 100mb of audit to a file per minute... and configuring rsyslogd to read AND configuring logrotate to rotate the logs fast enough is fairly challenging.

greenlitdesign commented 10 years ago

Hi, Is there any tools out there to parse Mysql McAfee audit audit file?

ruckc commented 10 years ago

Logstash -> elasticsearch -> Kibana On Mar 27, 2014 7:07 PM, "greenlitdesign" notifications@github.com wrote:

Hi, Is there any tools out there to parse Mysql McAfee audit audit file?

— Reply to this email directly or view it on GitHubhttps://github.com/mcafee/mysql-audit/issues/14#issuecomment-38872135 .

greenlitdesign commented 10 years ago

Thanks @ ruck.

I found this https://github.com/danmandle/JSON2CSV but it does not work

http://www.wesdeviers.net/the-mcafee-mysql-auditing-plugin (towards end of article, link to a python script).

ruckc commented 10 years ago

you want logstash...

On Thu, Mar 27, 2014 at 7:39 PM, greenlitdesign notifications@github.comwrote:

I found this https://github.com/danmandle/JSON2CSV but it does not work

http://www.wesdeviers.net/the-mcafee-mysql-auditing-plugin (towards end of article, link to a python script).

— Reply to this email directly or view it on GitHubhttps://github.com/mcafee/mysql-audit/issues/14#issuecomment-38874540 .

glicht commented 9 years ago

For everyone still arriving at this issue looking for a syslog solution, I am aware of the limitations when trying to relay via rsyslog. I plan to review this in the next few weeks and see how the plugin can be improved to work with rsyslog too. For now my recommendation it to use syslog-ng. To help with setup, I've put together a blog post about configuring the plugin to work with syslog-ng: http://lichtman.io/mcafee-mysql-audit-plugin-logging-to-syslog-ng/ .

h0nIg commented 9 years ago

any progress on this issue? :)

glicht commented 9 years ago

No news yet. But I can say that this is on the todo list...

Currently the way to go is to use syslog-ng as specified above.