ottobackwards
commented
2 years ago If you just run zeek -NN does it work?
Open clopmz opened 2 years ago
ottobackwards
commented
2 years ago If you just run zeek -NN does it work?
clopmz
commented
2 years ago Yes. Here it is: ...... Zeek::XMPP - XMPP analyzer (StartTLS only) (built-in) [Analyzer] XMPP (ANALYZER_XMPP, enabled) [Event] xmpp_starttls
Zeek::ZIP - Generic ZIP support analyzer (built-in) [Analyzer] ZIP (enabled)
Corelight::CommunityID - "Community ID" flow hash support in the connection log (dynamic, version 3.2.0) [Function] CommunityID::hash_conn
Seiso::Kafka - Writes logs to Kafka (dynamic, version 0.3.0) [Writer] KafkaWriter (Log::WRITER_KAFKAWRITER) [Constant] Kafka::kafka_conf [Constant] Kafka::additional_message_values [Constant] Kafka::topic_name [Constant] Kafka::max_wait_on_shutdown [Constant] Kafka::tag_json [Constant] Kafka::json_timestamps [Constant] Kafka::debug [Constant] Kafka::mock [Event] kafka_topic_resolved_event
Zeek::Netmap - Packet acquisition via Netmap (dynamic, version 1.0.0) [Packet Source] NetmapReader (interface prefix "netmap"; supports live input) [Packet Source] NetmapReader (interface prefix "vale"; supports live input)
Also installation goes well: root@fbsdzeekmgmt:~ # zpkg install zeek/seisollc/zeek-kafka The following packages will be INSTALLED: zeek/seisollc/zeek-kafka (v1.0.0)
Verify the following REQUIRED external dependencies: (Ensure their installation on all relevant systems before proceeding): from zeek/seisollc/zeek-kafka (v1.0.0): librdkafka ~1.4.2-RC1
Proceed? [Y/n] y zeek/seisollc/zeek-kafka asks for LIBRDKAFKA_ROOT (Path to librdkafka installation tree root) ? [/usr/local] Saved answers to config file: /opt/zeek/etc/zkg/config Running unit tests for "zeek/seisollc/zeek-kafka" Installing "zeek/seisollc/zeek-kafka".............................. Installed "zeek/seisollc/zeek-kafka" (v1.0.0) Loaded "zeek/seisollc/zeek-kafka"
ottobackwards
commented
2 years ago I'm at a loss, I don't know, unless it is deployed incorrectly across the cluster I'm not sure why it would fail.
If you can zeek -NN on each of the cluster nodes, then it was deployed correctly. to all nodes.
I'm asking on Slack as you can see there
clopmz
commented
2 years ago Good morning,
One thing: "zeek -NN" only works on the manager but not in the workers:
root@fbsdnsm01:/opt/zeek/lib/zeek/plugins/packages # /opt/zeek/bin/zeek -NN internal error: Failed to find variable named: Kafka::kafka_conf Abort root@fbsdnsm01:/opt/zeek/lib/zeek/plugins/packages #
JonZeolla
commented
2 years ago @clopmz it looks like you're running an old version of the package, can you attempt an update?
clopmz
commented
2 years ago Hi @JonZeolla ,
Uhmm ... older? Release installed in all zeek workers and manager are 1.8.2 ... According to https://github.com/edenhill/librdkafka/releases, latest release is 1.9.0 released 6 days ago ...
I will try it .... but I have serious doubts that this is it.
clopmz
commented
2 years ago Oops ... sorry ... My zeek-kafka package is release 1.0.0, and 1.1.0-rc1 was released 19 hours ago .....
Summary of the issue
Kafka install plugin works ok but when I run "zeekctl deploy" returns the following error:
==== stderr.log error in /nsm/zeek/spool/installed-scripts-do-not-touch/site/packages/./zeek-kafka/./logs-to-kafka.zeek, line 25: unknown identifier logs_to_send, at or near "logs_to_send" internal error in /nsm/zeek/spool/installed-scripts-do-not-touch/site/packages/./zeek-kafka/./logs-to-kafka.zeek, line 25: Failed to find variable named: Kafka::kafka_conf /opt/zeek/share/zeekctl/scripts/run-zeek: line 110: 27422 Abort trap nohup "$myzeek" "$@"
Expected behavior
That it works
Steps to reproduce
zeek/corelight/zeek-community-id (installed: 3.2.1) - "Community ID" flow hash support in conn.log zeek/corelight/zeek-long-connections (installed: v1.2.0) - Find and log long-lived connections into a "conn_long" log. zeek/salesforce/hassh (installed: master) - HASSH is used to identify specific Client and Server SSH implementations. zeek/salesforce/ja3 (installed: master) - JA3 creates 32 character SSL client fingerprints and logs them as a field in ssl.log. zeek/zeek/zeek-netmap (installed: v2.0.0) - Packet source plugin that provides native Netmap support.
redef Kafka::tag_json = T; redef Kafka::send_all_active_logs = T; redef Kafka::topic_name = "zeek"; redef Kafka::kafka_conf = table( ["metadata.broker.list"] = "172.22.58.8:9092" );
Where applicable, consider providing a patch that uses the end to end testing environment.
Logs, errors, etc.
==== stderr.log error in /nsm/zeek/spool/installed-scripts-do-not-touch/site/packages/./zeek-kafka/./logs-to-kafka.zeek, line 25: unknown identifier logs_to_send, at or near "logs_to_send" internal error in /nsm/zeek/spool/installed-scripts-do-not-touch/site/packages/./zeek-kafka/./logs-to-kafka.zeek, line 25: Failed to find variable named: Kafka::kafka_conf /opt/zeek/share/zeekctl/scripts/run-zeek: line 110: 27422 Abort trap nohup "$myzeek" "$@"
Your environment