issue with latest code and broker

[biswajitutil]

Hi @georgemakrakis , I have compiled your latest code with zeek 6.1.0 and spicy v1.9.0. I am able to generate iec104.hlto successfully. But when I am using a zeek script to send connection info to another daemon over broker, I am not able see anything on the other side. Earlier it was working fine, for each event fired I was sending few information to the external daemon over broker interface. What can go wrong here? Interestingly, I can see logs being generated locally and also you can see PEER is connected message and parsed data in stdout.

[root@ge-zeek-61]# /usr/local/zeek/bin/zeek -Cr iec104.pcap /usr/local/zeek/share/zeek/site/utiltyx-iec104/main.zeek processing suspended PEER ADDED, [id=31cf555d-2de0-5e2a-a15a-6e8415a6e838, network=[address=127.0.0.1, bound_port=50003/tcp]] processing continued NEW, [orig_h=10.20.102.1, orig_p=46413/tcp, resp_h=10.20.100.108, resp_p=2404/tcp] APCI request, [orig_h=10.20.102.1, orig_p=46413/tcp, resp_h=10.20.100.108, resp_p=2404/tcp], 4, 0, 0, 1 TESTFR act APCI request, [orig_h=10.20.102.1, orig_p=46413/tcp, resp_h=10.20.100.108, resp_p=2404/tcp], 4, 0, 0, 2 TESTFR con APCI request, [orig_h=10.20.102.1, orig_p=46413/tcp, resp_h=10.20.100.108, resp_p=2404/tcp], 4, 1, 0, 0 STARTDT act APCI request, [orig_h=10.20.102.1, orig_p=46413/tcp, resp_h=10.20.100.108, resp_p=2404/tcp], 4, 2, 0, 0 STARTDT con APCI request, [orig_h=10.20.102.1, orig_p=46413/tcp, resp_h=10.20.100.108, resp_p=2404/tcp], 14, 0, 0, 0 APCI request, [orig_h=10.20.102.1, orig_p=46413/tcp, resp_h=10.20.100.108, resp_p=2404/tcp], 14, 0, 0, 0 APCI request, [orig_h=10.20.102.1, orig_p=46413/tcp, resp_h=10.20.100.108, resp_p=2404/tcp], 14, 0, 0, 0 APCI request, [orig_h=10.20.102.1, orig_p=46413/tcp, resp_h=10.20.100.108, resp_p=2404/tcp], 14, 0, 0, 0 APCI request, [orig_h=10.20.102.1, orig_p=46413/tcp, resp_h=10.20.100.108, resp_p=2404/tcp], 26, 1, 0, 0 STARTDT act APCI request, [orig_h=10.20.102.1, orig_p=46413/tcp, resp_h=10.20.100.108, resp_p=2404/tcp], 26, 1, 0, 0 STARTDT act

[georgemakrakis]

@biswajitutil I do not know neither what your zeek script nor your external daemon is doing. Can you provide more information on that? Are you trying to parse the stdout or the produced log files?

Also, can you please clarify what do you mean by saying that is was working fine earlier? Can you refer to a commit that was working before?

[biswajitutil]

I have added below fields (octet 1, 2 , 3, 4 as o1, o2, o3, o4) in your existing parser to access each octet and I have also extended your .evt file to include this fields. Finally used these in main.zeek as below and send connection info over the broker. ( added the chunk of code below)

iec.spicy file changes:

type Apci = unit { %byte-order = spicy::ByteOrder::Little;

The IEC-104 has the below start byte in its beginning

    start :  b"\x68" &convert=$$.to_uint(spicy::ByteOrder::Little);
    apduLen : uint8 &byte-order = spicy::ByteOrder::Little;
    # There can be four 8-bit control fields
    ctrl : bitfield(32) {
        not_i_type : 0;
        #Only of value when not_i_type == 1
        #mode: 0..1 if ( self.not_i_type == 1 );
        apci_type : 0..1;
        **o1 : 0..7;
        o2 : 8..15;**
        #Only used when I mode
        apci_tx : 1..15;
        u_start_dt : 2..3;
        u_stop_dt : 4..5;
        u_test_fr : 6..7;
        **o3 : 16..23;
        o4 : 24..31;**
        #Not used when we have U mode
        apci_rx : 17..31;
    };

iec104.evt changes

  event iec104::apci($conn, $is_orig, self.apduLen, self.ctrl.not_i_type, self.ctrl.apci_type, self.ctrl.apci_tx, self.ctrl.u_start_dt, self.ctrl.u_stop_dt, self.ctrl.u_test_fr, self.ctrl.apci_rx, **self.ctrl.o1, self.ctrl.o2, self.ctrl.o3, self.ctrl.o4**);

main.zeek changes

global iec_topic = "/topic/iec104"; global iec_parsed: event(c: connection);

event zeek_init() &priority=5 { ..... Broker::peer(addr_to_uri(127.0.0.1), 50003/tcp); .... } event iec104::apci(c: connection, is_orig : bool, apdu_len : count, not_i_type : count, apci_type : count, apci_tx : count, u_start_dt : count, u_stop_dt : count, u_test_fr : count, apci_rx : count, o1 : count, o2 : count, o3 : count, o4 : count) &priority=4 { ......

Broker::publish(iec_topic, iec_parsed, c);

Log::write(iec104::LOG, info);

.......

}

the parser is successfully compiled and runnind but getting bizarre behavior.

1. Can't able to see the published event in receiver side, but the logs have been generated fine. Earlier I was able to see the events in subscriber with the same changes.
2. I am using the pcap you keep in your testing directory, I am getting mqtt connection removal event (subscribed in a different topic, different port) from the pcap but not iec104 event, I am not sure how zeek parser is firing wrong event with that pcap file. Currently, I am using zeek 6.1 with spicy 1.9.0.

Please let me know if these changes can make the things wrong anyway.

Thanks Biswa

[georgemakrakis]

@biswajitutil I cannot debug those changes that you made since I do not have the full context of what you try to accomplish. If the event logs are generated just fine, I would assume that the issue might be either in the version that you are using or your changes. I would suggest that you try different versions of Zeek and Spicy. Or even checkout a previous commit of my parser to see if there was a change there that is causing this behavior that you see.

I am still not sure what do you mean with it was working fine earlier and that earlier you were able to see the events in subscriber with the same changes.

[georgemakrakis]

I am closing this for now due to inactivity.