search

certego / fw1-loggrabber

FW1-Loggrabber is a command-line tool to grab logfiles from remote Checkpoint devices using OPSEC LEA (Log Export API)
GNU General Public License v2.0
53 stars 35 forks source link

Difference between online and no-online log fields #15

Open syunusic opened 8 years ago

syunusic commented 8 years ago

When I use ONLINE_MODE="yes" this is what I get in most of the log lines: time=2016-06-02 00:59:17|action=accept|orig=192.168.100.66|i/f_dir=inbound|i/f_name=eth3|has_accounting=1|uuid=<574faf15,00000021,4264a8c0,c0000000>|product=VPN-1 & FireWall-1|rule=44|rule_uid={E9C085F7-9DE8-4893-B1AF-86D9D255537F}|rule_name=Salida Google Apps, Youtube|service_id=UDP-443|src=192.168.226.51|s_port=65193|dst=64.233.186.100|service=443|proto=udp|xlatesrc=131.0.54.4|xlatesport=52436|xlatedport=0|NAT_rulenum=internal|NAT_addtnl_rulenum=internal|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={6D5AC528-0F96-9540-86C2-C3AE325686BA};mgmt=salvatore1;date=1464787912;policy_name=Conf2014]|origin_sic_name=CN=Firewall1000,O=salvatore.domain.com.q4a2j9

When I use ONLINE_MODE="no", this is what I get from the SAME log line (watch the UUID): time=2016-06-02 00:59:17|action=accept|orig=192.168.100.66|i/f_dir=inbound|i/f_name=eth3|has_accounting=1|uuid=<574faf15,00000021,4264a8c0,c0000000>|product=VPN-1 & FireWall-1|rule=44|rule_uid={E9C085F7-9DE8-4893-B1AF-86D9D255537F}|rule_name=Salida Google Apps, Youtube|service_id=UDP-443|src=192.168.226.51|s_port=65193|dst=64.233.186.100|service=443|proto=udp|xlatesrc=131.0.54.4|xlatesport=52436|xlatedport=0|NAT_rulenum=internal|NAT_addtnl_rulenum=internal|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={6D5AC528-0F96-9540-86C2-C3AE325686BA};mgmt=salvatore1;date=1464787912;policy_name=Conf2014]|origin_sic_name=CN=Firewall1000,O=salvatore.domain.com.q4a2j9|start_time= 2Jun2016 0:59:17|segment_time= 2Jun2016 0:59:17|elapsed=0:00:00|packets=18|bytes=8002|client_inbound_packets=9|client_outbound_packets=9|server_inbound_packets=9|server_outbound_packets=9|client_inbound_bytes=2495|client_outbound_bytes=5507|server_inbound_bytes=5507|server_outbound_bytes=2495|client_inbound_interface=eth3|server_outbound_interface=eth2|__pos=7|__nsons=0|__p_dport=0

As you see, everything is the same until "origin_sic_name=CN=Firewall1000,O=salvatore.domain.com.q4a2j9". Why is this difference? Another thing is that there are some log lines that have this extra fields in both modes, but their values are totaly different. In the one with ONLINE_MODE="yes", the values are much smaller: ONLINE_MODE="yes": time=2016-06-02 04:28:26|action=accept|orig=192.168.100.66|i/f_dir=inbound|i/f_name=eth3|has_accounting=1|uuid=<574fe01a,00000016,4264a8c0,c0000000>|product=VPN-1 & FireWall-1|rule=44|rule_uid={E9C085F7-9DE8-4893-B1AF-86D9D255537F}|rule_name=Salida Google Apps, Youtube|service_id=UDP-443|src=192.168.240.75|s_port=63368|dst=209.85.239.142|service=443|proto=udp|xlatesrc=131.0.54.4|xlatesport=31249|xlatedport=0|NAT_rulenum=internal|NAT_addtnl_rulenum=internal|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={6D5AC528-0F96-9540-86C2-C3AE325686BA};mgmt=salvatore1;date=1464787912;policy_name=Conf2014]|origin_sic_name=CN=Firewall1000,O=salvatore.domain.com.q4a2j9|start_time= 2Jun2016 4:28:26|segment_time= 2Jun2016 4:28:26|elapsed=0:00:00|packets=5|bytes=1602|client_inbound_packets=5|client_outbound_packets=0|server_inbound_packets=1|server_outbound_packets=5|client_inbound_bytes=1602|client_outbound_bytes=0|server_inbound_bytes=30|server_outbound_bytes=1604|client_inbound_interface=eth3|server_inbound_interface=eth2|server_outbound_interface=Mgmt|__pos=2|__nsons=0|__p_dport=0

ONLINE_MODE="no": time=2016-06-02 04:28:26|action=accept|orig=192.168.100.66|i/f_dir=inbound|i/f_name=eth3|has_accounting=1|uuid=<574fe01a,00000016,4264a8c0,c0000000>|product=VPN-1 & FireWall-1|rule=44|rule_uid={E9C085F7-9DE8-4893-B1AF-86D9D255537F}|rule_name=Salida Google Apps, Youtube|service_id=UDP-443|src=192.168.240.75|s_port=63368|dst=209.85.239.142|service=443|proto=udp|xlatesrc=131.0.54.4|xlatesport=31249|xlatedport=0|NAT_rulenum=internal|NAT_addtnl_rulenum=internal|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={6D5AC528-0F96-9540-86C2-C3AE325686BA};mgmt=salvatore1;date=1464787912;policy_name=Conf2014]|origin_sic_name=CN=Firewall2000,O=salvatore.domain.com.q4a2j9|start_time= 2Jun2016 4:28:26|segment_time= 2Jun2016 4:28:26|elapsed=0:00:01|packets=4125|bytes=3803464|client_inbound_packets=1370|client_outbound_packets=2753|server_inbound_packets=2757|server_outbound_packets=1374|client_inbound_bytes=109950|client_outbound_bytes=3693428|server_inbound_bytes=3693548|server_outbound_bytes=110182|__pos=7|__nsons=0|__p_dport=0

Any clues on this?

syunusic commented 8 years ago

I did more research and I'm possitive the problem is with the ONLINE_MODE="yes" flag. When the log has has_accounting=1, only has the fileds related to bytes and amount of packets if ONLINE_MODE is set to "no". The problem is I need to process logs in realtime. Please help!

adepasquale commented 8 years ago

FW1-LogGrabber retrieves all available fields by default.

My guess is that fields like the number of bytes/packets sent/received are not known immediately, but only when the connection is terminated. This might be the reason why some fields are not present when using ONLINE_MODE="yes", or have a smaller value.

syunusic commented 8 years ago

I thought the same, but If I take the SAME log line, first as online and then as not online, the second one has the account ones. It's weird, but is very consistent. Almost 99,9% of the time happens this way. This leads me to a question: nobody has online reporting with # of bytes or # of packets? I think that's the most important part of having firewall log s processed.

adepasquale commented 8 years ago

I have only used online mode without considering the accounting part, so I'm sorry but I can't be of any help. Have you considered asking Checkpoint directly?