activecm / ipfix-rita

Collect IPFIX / Netflow v9 Records and Ship them to RITA for Analysis
https://www.activecountermeasures.com/
10 stars 1 forks source link

Log Rollover Issue #65

Closed SamuelCarroll closed 5 years ago

SamuelCarroll commented 5 years ago

When running IPFIX-RITA capturing logs starting at 4PM and going until about 6:15PM MST on December 6th I discovered that the logs rolled over and started saving to December 7th.

I believe this is caused by MST being UTC-07:00 and 5PM would be midnight on December 7th so the logs automatically rolled over starting around that time.

If possible we should have IPFIX-RITA auto-detect and to adjust it's time zone based on system time.

I discovered that we can take UTC 00:00 to local time using the following code

    rn := time.Now() //get the system time right now
    _, secDiff := rn.Zone() //get the time zone difference (in seconds)

    //When going from UTC to local just adding the difference in nanoseconds
    //  We multiply by 1000000000 because duration is an int64 measured in nanoseconds
    secDiffDur := int64(secDiff) * int64(1000000000)
    time := logTime.Add(time.Duration(secDiffDur))
SamuelCarroll commented 5 years ago

Easiest way to go is to use the time.Local command. Given a timestamp in utc just append .Local() and it converts it to local time. This should work best since we just append 8 characters to a couple lines.

Zalgo2462 commented 5 years ago

Could you export TZ="America/Denver" in your .profile on the machine running ipfix-rita and see if that fixes things?

https://community.intersystems.com/post/setting-tz-environment-variable-linux <- Guide for setting the TZ variable.

I'm thinking that IPFIX-RITA can't tell what timezone it is in (in the docker container).

The local timezone is set according to this function in go: https://github.com/golang/go/blob/eef0140137bae3bc059f598843b8777f9223fac8/src/time/zoneinfo_unix.go#L28

We might bind mount in /etc/localtime as an alternative if this fixes it. This seems to be a popular solution: https://serverfault.com/questions/683605/docker-container-time-timezone-will-not-reflect-changes

Zalgo2462 commented 5 years ago

I've reproduced the issue on my system. I will apply the above fix tonight and let you know what happens.

EDIT: Turns out the suggested solution (bind mounting /etc/localtime) can mess things up on Mac. I know some of our devs occasionally use Mac, and theres no reason IPFIX-RITA cannot run on Mac at the moment.

An alternative solution was found by rewriting the solution at https://github.com/docker/for-mac/issues/2396#issuecomment-358150983 to use more reliable standard unix utitlities.

We set the TZ variable in the docker container to "$(basename $(dirname $(readlink /etc/localtime)))/$(basename $(readlink /etc/localtime))"

This solution was found at https://stackoverflow.com/questions/8223170/bash-extracting-last-two-dirs-for-a-pathname