compile and using ZMQ plugin as source, plugin compiled failed

[khemendra0]

Hi everyone,

I am facing issue when try to compile and use zeroMQ plugin

I am try below steps:

1. download code from ekuper: git@github.com:lf-edge/ekuiper.git

2. enter inside ekuiper folder and run make cmd (build kuiper) It generate _build folder which internaly hase bin, log, plugins, etc folders

3. then compile zmq plugin: compile zmq plugin: i. enter inside folder extensions and build the plugin go build -trimpath --buildmode=plugin -o sources/Zmq.so sources/zmq/zmq.go

This generate Zmq.so file

the I copied Zmq.so file to build folder bin directory: ii. cp Zmq.so ../../_build/kuiper-1.3.0-linux-amd64/plugins/sources/

4. Last I start kuiper like below: cd _build/kuiper-1.3.0-linux-amd64/bin ./kuperd

5. now added streams with zmq source {"sql":"create stream demo () WITH ( datasource = \"Event\", FORMAT = \"json\", KEY = \"temperature\", TYPE = \"zmq\")"}

6. next adding a rule for filter and send data to MQTT broker

Rule adding failed when chekced log below is error for plugin compile: closing rule rule2 for error: cannot open /home/root1/My_Work/EdgeGallery/IIOT_Analaysis/kuiper/install_lfedge/ekuiper/_build/kuiper-1.3.0-linux-amd64/plugins/sources/Zmq.so: plugin.Open(\"/home/root1/My_Work/EdgeGallery/IIOT_Analaysis/kuiper/install_lfedge/ekuiper/_build/kuiper-1.3.0-linux-amd64/plugins/sources/Zmq\"): plugin was built with a different version of package github.com/lf-edge/ekuiper/pkg/api" file="server/ruleManager.go:110

[ngjaying]

@khemendra0 The problem is step 3, it is required to run the build command in the root path of eKuiper instead of extensions folder.

go build -trimpath -modfile extensions.mod --buildmode=plugin -o ./_build/$build/plugins/sources/Zmq.so extensions/sources/zmq/zmq.go

Please check the updated plugin tutorial for detail.

[khemendra0]

Thank you so much ngjaying , compile is working and I am able to add rule with zmq source :)

But when I testing with zmq source, i am not getting any data to kuiper.

1. my source is zmq
2. sink is mqtt broker

I tested by publisher code and it is fine but data in not getting by kuiper and not sent out to sink Kindly help

below are snapshot

[khemendra0]

I found the reason, it was problem in my application. I need to add Event topic to my publish data, then it was working Thanks a lot :)

[khemendra0]

Now I am trying to compile and use zmq plugin in docker container.

I followed below steps:

1. start conatiner: docker run -p 9081:9081 -d --name kuiper -e MQTT_SOURCEDEFAULTSERVERS=[tcp://broker.emqx.io:1883] lfedge/ekuiper:1.3.0
2. enter inside container "docker exec -it containerID bash"
3. in ekuiper folder, build kuiper make
4. in ekuiper folder, build zmq plugin: go build -trimpath -modfile extensions.mod --buildmode=plugin -o ./_build/kuiper-1.3.0-linux-amd64/plugins/sources/Zmq.so extensions/sources/zmq/zmq.go

I can see both kuiper "kuiperd" and zmq binary Zmq.so are generated succesuffuly.

But now I added stream and rule for source as zmq and sink as mqtt.

But it failed as zmq plugin not found

Kindly help

[ngjaying]

@khemendra0 It seems you need to restart the docker container to reload the plugins or use REST API/CLI to register the new plugin in runtime.

By the way, step 3 is not needed in the docker as it already built the eKuiper main program when creating the docker image. And it is supposed not to be changed to make sure it is exactly the same env as the production docker image.

[ngjaying]

@khemendra0 Additionally, we have pre-built zip file for shipped plugins like zmq. You can install it by REST API as below. Change the version and arch in the url to match your env.

curl -X POST --location "http://127.0.0.1:9081/plugins/sinks" \
    -H "Content-Type: application/json" \
    -d "{\"name\":\"zmq\", \"file\": \"https://packages.emqx.io/kuiper-plugins/1.3.0/debian/sinks/zmq_amd64.zip\"}"