Reduce the amount of data going to syslog

suganoo / s3fs

Automatically exported from code.google.com/p/s3fs

0 stars 0 forks source link

Reduce the amount of data going to syslog #120

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago

From the Wiki 

Comment by dirk.taggesell,

How can I make s3fs stop logging every single request? It is just clogging up 
my /var/log and while I could adjust my syslog.conf to re-route or throw away 
any s3fs log data, I would rather like to prevent these excessive log data to 
be produced at all. Only errors and other important stuff should be logged.

Original issue reported on code.google.com by dmoore4...@gmail.com on 7 Nov 2010 at 2:04

GoogleCodeExporter commented 8 years ago

An idea on how to accomplish this is to use a command line option (-d --debug) 
to turn on logging of LOG_DEBUG messages to syslog() (default would be not to 
log LOG_DEBUG messages).

Also, a review may be needed to recategorized some LOG_INFO messages to 
LOG_DEBUG

Original comment by dmoore4...@gmail.com on 11 Nov 2010 at 5:29

Changed state: Accepted

GoogleCodeExporter commented 8 years ago

This issue was closed by revision r235.

Original comment by dmoore4...@gmail.com on 12 Nov 2010 at 11:18

Changed state: Fixed

GoogleCodeExporter commented 8 years ago

Logging to /var/log/syslog AND  /var/log/messages still happening in Ubuntu 
Jaunty in R271.

/usr/local/bin/s3fs bucket /s3 -o default_acl=public-read;

Original comment by gazzet...@gmail.com on 3 Dec 2010 at 7:14

GoogleCodeExporter commented 8 years ago

This is a different issue and not the one originally cited. Dirk's issue was 
that there was an excessive amount of log data.  That has been minimized, but 
not totally eliminated.

Your issue seems to be that log messages are going to both /var/log/syslog and 
/var/log/messages?  If so, I believe that that's not a function of s3fs but how 
your system is configured.

As to what the original issue was, are you still seeing excessive log messages 
going to /var/log/syslog?  If so, please provide a sampling of these messages. 
Thank you.

Original comment by dmoore4...@gmail.com on 3 Dec 2010 at 9:18

GoogleCodeExporter commented 8 years ago

Ubuntu defaults to syslog AND messages .. will get this fixed locally.

Problem is with the amount of data. 3 million jpg files being moved to s3.

Each jpg has the following entry into both log files.

Dec  4 13:14:10 domU-12-31-39-00-74-81 s3fs: LOCAL FD
Dec  4 13:14:10 domU-12-31-39-00-74-81 s3fs: copy 
path=/portrait_small/.0128natasha.jpg.GRCHcz
Dec  4 13:14:10 domU-12-31-39-00-74-81 s3fs: upload 
path=/portrait_small/.0128natasha.jpg.GRCHcz size=6468
Dec  4 13:14:10 domU-12-31-39-00-74-81 s3fs: copy 
path=/portrait_small/.0128natasha.jpg.GRCHcz
Dec  4 13:14:11 domU-12-31-39-00-74-81 s3fs: last message repeated 2 times
Dec  4 13:14:11 domU-12-31-39-00-74-81 s3fs: copy 
path=/portrait_small/0128natasha.jpg

This is a lot of logging. Can this be switched off?

Original comment by gazzet...@gmail.com on 4 Dec 2010 at 1:17

GoogleCodeExporter commented 8 years ago

Re-opened.

Original comment by dmoore4...@gmail.com on 4 Dec 2010 at 8:06

Changed state: Started

GoogleCodeExporter commented 8 years ago

This issue was closed by revision r274.

Original comment by dmoore4...@gmail.com on 4 Dec 2010 at 8:07

Changed state: Fixed