@load packages/zeek-kafka can't find

puquanyang0326 commented 8 months ago

I installed zeek-kafka via Manual Installation and it successfully outputs as follows [root@securitypublicservicestest-bj-1 172.16.252.5 bin]# . /zeek -N Seiso::Kafka Seiso::Kafka - Writes logs to Kafka (dynamic, version 0.3.0)

I followed up by writing local.zeek by referring to the documentation, which looks like this @load packages/zeek-kafka redef Kafka::send_all_active_logs = T; redef Kafka::kafka_conf = table( ["metadata.broker.list"] = "localhost:9092" ).

At this point an error message is given [ZeekControl] > deploy checking configurations ... zeek scripts failed. fatal error in /opt/zeek/share/zeek/site/local.zeek, line 121: can't find packages/zeek-kafka

[ZeekControl] > quit

ottobackwards commented 8 months ago

try just @load packages

marvi commented 2 months ago

I have the same error. Compiled the plugin from source.

[root@zeek lib64]# zeek -N Seiso::Kafka
Seiso::Kafka - Writes logs to Kafka (dynamic, version 0.3.0)

[root@zeek lib64]# zeek --version
zeek version 6.0.3

zeek/site/local.zeek:

@load packages/zeek-kafka
redef Kafka::send_all_active_logs = T;
redef Kafka::kafka_conf = table(
    ["metadata.broker.list"] = "localhost:9092"
);

Error:

[root@zeek lib64]# zeekctl deploy
checking configurations ...
zeek scripts failed.
fatal error in /usr/local/zeek/share/zeek/site/local.zeek, line 124: can't find packages/zeek-kafka

Changed to @load packages

[root@zeek lib64]# zeekctl deploy
checking configurations ...
zeek scripts failed.
fatal error in /usr/local/zeek/share/zeek/site/local.zeek, line 124: can't find packages

ottobackwards commented 2 months ago

https://github.com/SeisoLLC/zeek-kafka/issues/80#issuecomment-1798933927

marvi commented 2 months ago

I did.

https://github.com/SeisoLLC/zeek-kafka/issues/80#issuecomment-2079460929

marvi commented 2 months ago

I did some more tests. I read in the Zeek documentation that "By default, Zeek will automatically activate all dynamic plugins found in its search path ZEEK_PLUGIN_PATH". So I remove all @load directives. Now zeek starts without error:

[root@zeek bin]# zeekctl deploy
checking configurations ...
installing ...
removing old policies in /usr/local/zeek/spool/installed-scripts-do-not-touch/site ...
removing old policies in /usr/local/zeek/spool/installed-scripts-do-not-touch/auto ...
creating policy directories ...
installing site policies ...
generating standalone-layout.zeek ...
generating local-networks.zeek ...
generating zeekctl-config.zeek ...
generating zeekctl-config.sh ...
stopping ...
stopping zeek ...
starting ...
starting zeek ...

zeek-kafka seems to be loaded:

[root@zeek current]# zeek -N Seiso::Kafka
Seiso::Kafka - Writes logs to Kafka (dynamic, version 0.3.0)

[root@zeek bin]# zeekctl scripts |grep kafka
  {"name":"  /usr/local/zeek/lib64/zeek/plugins/SEISO_KAFKA/lib/bif/kafka.bif.zeek"}

Kafka is accessible on localhost:9092:

[root@zeek bin]# ./kafka-broker-api-versions.sh --bootstrap-server localhost:9092
zeek.marvi.xyz:9092 (id: 1 rack: null) -> (
        Produce(0): 0 to 10 [usable: 10],
        Fetch(1): 0 to 16 [usable: 16],
        ListOffsets(2): 0 to 8 [usable: 8],

Configuration:

redef Kafka::send_all_active_logs = T;
redef Kafka::topic_name = "zeek";
redef Kafka::kafka_conf = table(
    ["metadata.broker.list"] = "localhost:9092"
);

Nothing is produced on the zeek topic. logs/current/ is populated.

I compiled zeek with debug enabled and recompiled the plugin. Then starting zeek aszeek -B plugin-Seiso-Kafka. debug.log is empty. Is there any other way to get debug information so I can track down my issue?

SeisoLLC / zeek-kafka

@load packages/zeek-kafka can't find #80