Events log position

[fbetil]

Hi, In v0.5.3, i have declared and event on a rotated log file (ie mylog-Ymd.log and mylog.log for current logfile). It seem that sometime the position is lost and logsniffer resend old events.. Thanks a lot, Florian

[mbok]

Hi Florian,

Could you please provide some details for trying to isolate the problem? When you say sometime, does it mean you have already seen that issue more then once? Can you see a pattern, when or under which circumstances the error occurs? Does it look like a timing issue e.g. when the files got rolled? Any hint would help. Is it possible that you provide the sequence of the rolled log files as they are listed on the "Info" page available from the context menu in the log viewer?

In general logsniffer navigates in rolled log files very stable. The only case I can imagine for such an issue is, when the attribute "Order criteria for rolled files" in the source configuration is set to "Time of last modification" and over the time the rolled log files will get modified. Thus the order and the navigation will be damaged and an event could be detected twice. Could it be the cause in your case?

Best, Michael

[fbetil]

Hi, I had temporaly pause the scanner, so i have reset the position to the end and reactivate the scanner. The next time it happen, i send you my config and the logs. Thank you, Florian

[fbetil]

Hi,

The configuration for the log:

Source type: Rolling log file with dynamic live file name Path pattern: /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7*.log Order criteria: File name Resolved logs: /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7.log (256 MB) Log parts: /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7.log (784 KB) /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7-20160322.log (8 MB) /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7-20160321.log (13 MB) /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7-20160320.log (3 MB) /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7-20160318.log (10 MB) /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7-20160317.log (10 MB) /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7-20160316.log (11 MB) /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7-20160315.log (11 MB) /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7-20160314.log (13 MB) /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7-20160313.log (3 MB) /mnt/rachel/D$/APPLICATIFS/INTERFACES/HL7_CLIENT/IntClientHL7-20160311.log (10 MB) ...

The folder /mnt/rachel/D$ is a mounted drive on a Windows 2008 R2 server (autofs)

The configuration for the scanner:

Scanner type: Regular expression scanner Source field: message Pattern: ORA-\d{5}

Before starting scanner, i set the position to the end, but yesterday, 10:19pm, logsniffer throw by email this event, that have occured at 2/22/2016 11:00am :

Event link: http://leanie:8888/c/sniffers/2/events/#/AVOlV23VbV3jjhqjX28N Log entries: *\ 22/02/2016 11:00:10 : Erreur ORACLE [-1017] detectee : [ORA-01017: invalid username/password; logon denied ]

When i go to the status page of the sniffer, i see that the position is now on an old log file.

Thank for your help.

Florian

[mbok]

Thank you for the detail information! Could you please look into logsniffer's own log for the time period around 10:19pm yesterday and check for warnings or errors. It would be very helpfull.

Thanks in advance! Michael

[fbetil]

This is the log for yesterday.Thanks

Date: Thu, 24 Mar 2016 06:07:12 -0700 From: notifications@github.com To: logsniffer@noreply.github.com CC: fbetil@free.fr Subject: Re: [logsniffer] Events log position (#61)

Thank you for the detail information! Could you please look into logsniffer's own log for the time period around 10:19pm yesterday and check for warnings or errors. It would be very helpfull.

Thanks in advance!

Michael

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub

[mbok]

Hi Florian,

the provided logs don't contain errors or hints for an application error. Thus the single cause I've in mind is that the running sniffer resets the position to the beginning or to a last safe point in the past logs when the determined list of rolled logs has got changed drastically. In you case the logs are read from a remote store mounted via autofs. Could in some circumstances the list of resolved log files be for a moment incomplete? Probably when the directory is unmounted because of no use and auto-mounted again when logsniffer tries to resolve and read the logs. I've also read about the -ghost option in autofs, but not sure if that correlates to the error scenario.

Summarized my speculations for now are around something to do with autofs and incomplete resolution of log files during an active sniffer run. As an interim workaround I'd suggest to switch to another operating concept for logsniffer to access the log files. A possible solution could be to transfer the logs e.g. via syslog-ng (see my blog post http://www.logsniffer.com/central-log-management-recipe/) to a host where logsniffer is running. For the coming 0.6 feature release I'll add some more logs to the significant application components in order to trace your scenario in a detail way.

Best, Michael

[fbetil]

Hi.I was on hollidays..I'll wait for the 0.6 release and i'll send logs to you.Thanks;Florian

Date: Thu, 7 Apr 2016 15:04:59 -0700 From: notifications@github.com To: logsniffer@noreply.github.com CC: fbetil@free.fr Subject: Re: [logsniffer/logsniffer] Events log position (#61)

Hi Florian,

the provided logs don't contain errors or hints for an application error. Thus the single cause I've in mind is that the running sniffer resets the position to the beginning or to a last safe point in the past logs when the determined list of rolled logs has got changed drastically. In you case the logs are read from a remote store mounted via autofs. Could in some circumstances the list of resolved log files be for a moment incomplete? Probably when the directory is unmounted because of no use and auto-mounted again when logsniffer tries to resolve and read the logs. I've also read about the -ghost option in autofs, but not sure if that correlates to the error scenario.

Summarized my speculations for now are around something to do with autofs and incomplete resolution of log files during an active sniffer run. As an interim workaround I'd suggest to switch to another operating concept for logsniffer to access the log files. A possible solution could be to transfer the logs e.g. via syslog-ng (see my blog post http://www.logsniffer.com/central-log-management-recipe/) to a host where logsniffer is running.

For the coming 0.6 feature release I'll add some more logs to the significant application components in order to trace your scenario in a detail way.

Best,

Michael

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub