search

pebbe / zmq4

A Go interface to ZeroMQ version 4
BSD 2-Clause "Simplified" License
1.17k stars 163 forks source link

ZMQ version mismatch #126

Closed omani closed 6 years ago

omani commented 6 years ago

Hi pebbe, I am inclined to say that something is broken with the way pebbe/zqm4 is checking for zmq lib versions or I am really overlooking something here. any help is appreciated.

please have a look at this. I will give you the whole workflow I do here.

let's start with a clean go environment:

~  $ rm -rf gocode/src/github.com/pebbe/
~  $ rm -rf gocode/pkg/
~  $ go get -u github.com/pebbe/zmq4
# pkg-config --cflags libzmq
Package libzmq was not found in the pkg-config search path.
Perhaps you should add the directory containing `libzmq.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libzmq' found
pkg-config: exit status 1
~  $

so far so good. zmq4 is missing libzmq package. which is right. because I have none installed yet. so let's do that:

~  $ sudo apt-cache policy libczmq-dev
libczmq-dev:
  Installed: (none)
  Candidate: 4.1.0
  Version table:
     4.1.0 500
        500 http://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-stable/Debian_9.0 ./ Packages
     3.0.2-5 500
        500 http://de.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
~  $

notice that I am downloading from official zmq repository for debian9.0/ubuntu like said on http://czmq.zeromq.org/page:get-the-software

now let's check if we really have no libzmq libs installed:

~  $ ll /usr/lib/x86_64-linux-gnu/libzmq*
ls: cannot access '/usr/lib/x86_64-linux-gnu/libzmq*': No such file or directory

now let's install libczmq-dev finally:

~  $ sudo apt-get install libczmq-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libczmq4 libnorm1 libsodium18 libsystemd-dev libzmq3-dev libzmq5
The following NEW packages will be installed:
  libczmq-dev libczmq4 libnorm1 libsodium18 libsystemd-dev libzmq3-dev libzmq5
0 upgraded, 7 newly installed, 0 to remove and 178 not upgraded.
Need to get 0 B/1,331 kB of archives.
After this operation, 5,340 kB of additional disk space will be used.
Do you want to continue? [Y/n] 
Selecting previously unselected package libnorm1:amd64.
(Reading database ... 237362 files and directories currently installed.)
Preparing to unpack .../libnorm1_1.5r6+dfsg1-1_amd64.deb ...
Unpacking libnorm1:amd64 (1.5r6+dfsg1-1) ...
Selecting previously unselected package libsodium18:amd64.
Preparing to unpack .../libsodium18_1.0.8-5_amd64.deb ...
Unpacking libsodium18:amd64 (1.0.8-5) ...
Selecting previously unselected package libzmq5:amd64.
Preparing to unpack .../libzmq5_4.2.3_amd64.deb ...
Unpacking libzmq5:amd64 (4.2.3) ...
Selecting previously unselected package libzmq3-dev:amd64.
Preparing to unpack .../libzmq3-dev_4.2.3_amd64.deb ...
Unpacking libzmq3-dev:amd64 (4.2.3) ...
Selecting previously unselected package libsystemd-dev:amd64.
Preparing to unpack .../libsystemd-dev_229-4ubuntu21.1_amd64.deb ...
Unpacking libsystemd-dev:amd64 (229-4ubuntu21.1) ...
Selecting previously unselected package libczmq4.
Preparing to unpack .../libczmq4_4.1.0_amd64.deb ...
Unpacking libczmq4 (4.1.0) ...
Selecting previously unselected package libczmq-dev.
Preparing to unpack .../libczmq-dev_4.1.0_amd64.deb ...
Unpacking libczmq-dev (4.1.0) ...
Processing triggers for libc-bin (2.23-0ubuntu10) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up libnorm1:amd64 (1.5r6+dfsg1-1) ...
Setting up libsodium18:amd64 (1.0.8-5) ...
Setting up libzmq5:amd64 (4.2.3) ...
Setting up libzmq3-dev:amd64 (4.2.3) ...
Setting up libsystemd-dev:amd64 (229-4ubuntu21.1) ...
Setting up libczmq4 (4.1.0) ...
Setting up libczmq-dev (4.1.0) ...
Processing triggers for libc-bin (2.23-0ubuntu10) ...
~  $

check again:

~  $ ll /usr/lib/x86_64-linux-gnu/libzmq*
-rw-r--r-- 1 root root 1.8M Dec 31  2014 /usr/lib/x86_64-linux-gnu/libzmq.a
lrwxrwxrwx 1 root root   15 Dec 31  2014 /usr/lib/x86_64-linux-gnu/libzmq.so -> libzmq.so.5.1.3
lrwxrwxrwx 1 root root   15 Dec 31  2014 /usr/lib/x86_64-linux-gnu/libzmq.so.5 -> libzmq.so.5.1.3
-rw-r--r-- 1 root root 596K Dec 31  2014 /usr/lib/x86_64-linux-gnu/libzmq.so.5.1.3
~  $

5.1.3 stands for version 4.2.3 here.

now let's see what pkgconfig would say to pebbe/zmq4 if pebbe/zmq4 were to ask for it when we go get this repository:

~  $ cat /usr/lib/x86_64-linux-gnu/pkgconfig/libzmq.pc 
prefix=/usr
exec_prefix=${prefix}
libdir=${prefix}/lib/x86_64-linux-gnu
includedir=${prefix}/include

Name: libzmq
Description: 0MQ c++ library
Version: 4.2.3
Libs: -L${libdir} -lzmq
Libs.private: -lstdc++  -lsodium -lpgm -lpthread -lm -lnorm
Cflags: -I${includedir} 
~  $

notice: it says you will find the libzmq packages/libs in /usr/lib/x86_64-linux-gnu/ with version 4.2.3.

so far so good. let's continue to go get this repository finally:

~  $ go get -u github.com/pebbe/zmq4
~  $

worked. now let's continue to build our go application:

~/gocode/src/github.com/omani/lol/cmd (master [uncommited!]) $ go build -o lol main.go
~/gocode/src/github.com/omani/lol/cmd (master [uncommited!]) $

worked. now let's run the just created binary:

~/gocode/src/github.com/omani/lol/cmd (master [uncommited!]) $ ./lol

USAGE:
   lol -c=<string> --identity=<string>
                    [--debug]
   lol --help
   lol --version

Use 'lol --help' to get detailed information about options and examples of usage.
~/gocode/src/github.com/omani/lol/cmd (master [uncommited!]) $

so at first sight, it seems to work. but that's just an illusion. this is only flags doing it's job and returning due to missing arguments. we never really ran the app. so now let's give the application some working cli arguments to see what really happens:

~/gocode/src/github.com/omani/lol/cmd (master [uncommited!]) $ ./lol -c ~/lol.toml -identity lol01 -debug
2018/03/10 15:27:57 zmq4 was installed with ZeroMQ version 4.2.4, but the application links with version 4.2.3
~/gocode/src/github.com/omani/lol/cmd (master [uncommited!]) $

there you go. the app does not start because it's complaining about zmq.

now here is my question: How can this even be? As you can see we installed libczmq version 4.2.3. and all it's dependencies for version 4.2.3. Why should this ever happen that zmq4 is thinking we had version 4.2.4 during the time we ran go get github.com/pebbe/zmq4? You see what pkgconfig said which version we have installed and I assume that is the same information zmq4 is getting.

so where does version 4.2.4 come from?

thanks.

omani commented 6 years ago

also note: I have tried this on every possible machine I have available here. All ubuntu machines. same outcome.

~  $ uname -a
Linux machine01 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

~  $ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:    16.04
Codename:   xenial
~  $
pebbe commented 6 years ago

Do you have experimental repositories added to your apt-get? Ubuntu Xenial has ZeroMQ version 4.1.4 which is installed with package libzmq3-dev (not libczmq-dev).

Here is my run with Xenial (in Docker) with Go installed, but without ZeroMQ:

[docker:xenial] root:~# export GOPATH=$HOME/go
[docker:xenial] root:~# go get github.com/pebbe/zmq4
# pkg-config --cflags libzmq
Package libzmq was not found in the pkg-config search path.
Perhaps you should add the directory containing `libzmq.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libzmq' found
pkg-config: exit status 1
[docker:xenial] root:~# apt-get install libzmq3-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libsodium18 libzmq5
The following NEW packages will be installed:
  libsodium18 libzmq3-dev libzmq5
0 upgraded, 3 newly installed, 0 to remove and 24 not upgraded.
Need to get 575 kB of archives.
After this operation, 2,320 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://archive.ubuntu.com/ubuntu xenial/universe amd64 libsodium18 amd64 1.0.8-5 [144 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial/universe amd64 libzmq5 amd64 4.1.4-7 [149 kB]
Get:3 http://archive.ubuntu.com/ubuntu xenial/universe amd64 libzmq3-dev amd64 4.1.4-7 [282 kB]
Fetched 575 kB in 0s (1,523 kB/s)    
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libsodium18:amd64.
(Reading database ... 32115 files and directories currently installed.)
Preparing to unpack .../libsodium18_1.0.8-5_amd64.deb ...
Unpacking libsodium18:amd64 (1.0.8-5) ...
Selecting previously unselected package libzmq5:amd64.
Preparing to unpack .../libzmq5_4.1.4-7_amd64.deb ...
Unpacking libzmq5:amd64 (4.1.4-7) ...
Selecting previously unselected package libzmq3-dev:amd64.
Preparing to unpack .../libzmq3-dev_4.1.4-7_amd64.deb ...
Unpacking libzmq3-dev:amd64 (4.1.4-7) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up libsodium18:amd64 (1.0.8-5) ...
Setting up libzmq5:amd64 (4.1.4-7) ...
Setting up libzmq3-dev:amd64 (4.1.4-7) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...
[docker:xenial] root:~# go get github.com/pebbe/zmq4
[docker:xenial] root:~# cd go/src/github.com/pebbe/zmq4/examples
[docker:xenial] root:~/go/src/github.com/pebbe/zmq4/examples# go build version.go 
[docker:xenial] root:~/go/src/github.com/pebbe/zmq4/examples# ./version 
Current 0MQ version is 4.1.4
[docker:xenial] root:~/go/src/github.com/pebbe/zmq4/examples# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.3 LTS
Release:        16.04
Codename:       xenial
omani commented 6 years ago

my apt-get install gets every package from the imported opensuse sources.list as shown in my opening post:

~  $ dpkg -l | grep zmq
ii  libczmq-dev                                4.1.0                                        amd64        czmq development tools
ii  libczmq4                                   4.1.0                                        amd64        czmq shared library
ii  libzmq3-dev:amd64                          4.2.3                                        amd64        lightweight messaging kernel (development files)
ii  libzmq5:amd64                              4.2.3                                        amd64        lightweight messaging kernel (shared library)
~  $ 
~  $ apt-cache policy libczmq-dev
libczmq-dev:
  Installed: 4.1.0
  Candidate: 4.1.0
  Version table:
 *** 4.1.0 500
        500 http://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-stable/Debian_9.0 ./ Packages
        100 /var/lib/dpkg/status
     3.0.2-5 500
        500 http://de.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages

~  $ apt-cache policy libzmq5
libzmq5:
  Installed: 4.2.3
  Candidate: 4.2.3
  Version table:
 *** 4.2.3 500
        500 http://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-stable/Debian_9.0 ./ Packages
        100 /var/lib/dpkg/status
     4.1.4-7 500
        500 http://de.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages

~  $ apt-cache policy libzmq3-dev
libzmq3-dev:
  Installed: 4.2.3
  Candidate: 4.2.3
  Version table:
 *** 4.2.3 500
        500 http://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-stable/Debian_9.0 ./ Packages
        100 /var/lib/dpkg/status
     4.1.4-7 500
        500 http://de.archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
~  $

so to answer your question:

Do you have experimental repositories added to your apt-get?

no as you can see above, I am only using one repository. and that is the official zmq repository hosted on opensuse.

also: if we run go get github.com/pebbe/zmq4 we can say the following happens:

When you did go get github.com/pebbe/zmq4 the header file zmq.h of ZeroMQ was for version 4.2.1.

is what you once told me in another issue #124. so in this example, we should find version 4.2.3 in the header file, right? let's see.

so if I check that header file for version info I see the following:

~  $ grep -Er "VERSION" /usr/include/zmq.h
#define ZMQ_VERSION_MAJOR 4
#define ZMQ_VERSION_MINOR 2
#define ZMQ_VERSION_PATCH 3
#define ZMQ_MAKE_VERSION(major, minor, patch) \
#define ZMQ_VERSION \
    ZMQ_MAKE_VERSION(ZMQ_VERSION_MAJOR, ZMQ_VERSION_MINOR, ZMQ_VERSION_PATCH)
#define ZMQ_PROTOCOL_ERROR_ZAP_BAD_VERSION 0x20000003
~  $

if zmq4 is truly checking the header file, it should see version 4.2.3 as defined in zmq.h.

I can't find any package referencing version 4.2.4 on my system.

I think you should be able to reproduce my outcome if you follow the bash invocations of my opening post.

omani commented 6 years ago

also, when I do what you did: running the version example in this repository, I get the following output:

~/gocode/src/github.com/pebbe/zmq4/examples (master) $ go build version.go 
~/gocode/src/github.com/pebbe/zmq4/examples (master [uncommited!]) $ ./version 
Current 0MQ version is 0.0.0
pebbe commented 6 years ago

Why are you downloading from opensuse on Ubuntu?

[docker:xenial] root:~# apt-cache policy libczmq-dev
libczmq-dev:
  Installed: (none)
  Candidate: 3.0.2-5
  Version table:
     3.0.2-5 500
        500 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages
[docker:xenial] root:~# apt-cache policy libzmq3-dev
libzmq3-dev:
  Installed: (none)
  Candidate: 4.1.4-7
  Version table:
     4.1.4-7 500
        500 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages

You need libzmq3-dev, not libczmq-dev

omani commented 6 years ago

Why are you downloading from opensuse on Ubuntu?

because the ubuntu packages are too old for my application. I need at least version >4.2. ubuntu by default comes with version 4.1.

You need libzmq3-dev, not libczmq-dev

but libzmq3-dev is installed as I have shown in the previous comments:

 $ dpkg -l | grep libzmq3-dev
ii  libzmq3-dev:amd64                          4.2.3                                        amd64        lightweight messaging kernel (development files)
$

the only difference is, that my version of libzmq3-dev is newer than yours (because you are using the ubuntu repos. I am using zmq's official repos, providing me with >4.2v).

omani commented 6 years ago

I just did what you did. I removed the opensuse from my repos and installed zmq from ubuntu. only libzmq3-dev.

now I get the following error when building my application:

~/gocode/src/github.com/omani/lol/cmd (master [uncommited!]) $ go build -o lol main.go 
# command-line-arguments
/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/tmp/go-link-343529835/000000.o: In function `_cgo_5202562de717_C2func_zmq_curve_public':
/tmp/go-build/github.com/pebbe/zmq4/_obj/cgo-gcc-prolog:44: undefined reference to `zmq_curve_public'
/tmp/go-link-343529835/000000.o: In function `_cgo_5202562de717_Cfunc_zmq_curve_public':
/tmp/go-build/github.com/pebbe/zmq4/_obj/cgo-gcc-prolog:77: undefined reference to `zmq_curve_public'
collect2: error: ld returned 1 exit status

so if you want to know the reason why I need to use version >4.2v of zmq. I am using curve zmq. which ubuntu does not provide. ubuntu packages are too old.

~  $ grep -r VERSION /usr/include/zmq.h
#define ZMQ_VERSION_MAJOR 4
#define ZMQ_VERSION_MINOR 1
#define ZMQ_VERSION_PATCH 4
#define ZMQ_MAKE_VERSION(major, minor, patch) \
#define ZMQ_VERSION \
    ZMQ_MAKE_VERSION(ZMQ_VERSION_MAJOR, ZMQ_VERSION_MINOR, ZMQ_VERSION_PATCH)
~  $

this is the header file with ubuntu's packages.

omani commented 6 years ago

oh pebbe, look what I've found:

~  $ grep VERSION /usr/include/zmq.h
#define ZMQ_VERSION_MAJOR 4
#define ZMQ_VERSION_MINOR 2
#define ZMQ_VERSION_PATCH 3
#define ZMQ_MAKE_VERSION(major, minor, patch) \
#define ZMQ_VERSION \
    ZMQ_MAKE_VERSION(ZMQ_VERSION_MAJOR, ZMQ_VERSION_MINOR, ZMQ_VERSION_PATCH)
#define ZMQ_PROTOCOL_ERROR_ZAP_BAD_VERSION 0x20000003

~  $ grep VERSION /usr/local/include/zmq.h
#define ZMQ_VERSION_MAJOR 4
#define ZMQ_VERSION_MINOR 2
#define ZMQ_VERSION_PATCH 4
#define ZMQ_MAKE_VERSION(major, minor, patch)                                  \
#define ZMQ_VERSION                                                            \
    ZMQ_MAKE_VERSION (ZMQ_VERSION_MAJOR, ZMQ_VERSION_MINOR, ZMQ_VERSION_PATCH)
#define ZMQ_PROTOCOL_ERROR_ZAP_BAD_VERSION 0x20000003
omani commented 6 years ago

I have found another zmq.h header file in /usr/local/include. I guess this is coming from back then when I was building zmq from source (I had indeed built zmq version 4.2.4 once back then).

so it ended up in /usr/local/include. the package repos place the header file in /usr/include instead.

so what does this mean for pebb/zmq4. does it check for header files in /usr/local/include first and ignoring the rest after finding the header file there? is the order something like:

if so, then it is my fault for placing another header file there with v4.2.4 into /usr/local/include which takes precedence over /usr/include.

omani commented 6 years ago

I don't know the internals of how the C toolchain works. but I guess pkgconfig and the like have contributed to this. so basically the zmq.h in /usr/local/include was reported to pebbe/zmq4 during go get.

after removing the zmq.h file in /usr/local/include and having the zmq.h file in /usr/include placed by the opensuse repo, now it works.

I no more get the error:

zmq4 was installed with ZeroMQ version 4.2.4, but the application links with version 4.2.3

thank you pebbe for helping me find the right path. I hope this issue helps people in future and shows how important it is to check twice for things. especially header files :)

thanks.

pebbe commented 6 years ago

With go get github.com/pebbe/zmq4 all it does is this:

pkg-config --cflags libzmq
pkg-config --libs libzmq

If these commands don't list a path to the header or library files, then it's up to the build system to find the right location. If libzmq.so.5 and zmq.h are not for the same version, you get in trouble. The question is, when does this happen? During installation of zmq4, or during building of your application?

By the way, when I build a curve application on Xenial, it works fine:

[docker:xenial] root:~# cd go/src/github.com/pebbe/zmq4/examples_security
[docker:xenial] root:~/go/src/github.com/pebbe/zmq4/examples_security# make
go build grasslands.go
go build strawhouse.go
go build woodhouse.go
go build stonehouse.go
go build ironhouse.go
[docker:xenial] root:~/go/src/github.com/pebbe/zmq4/examples_security# ./ironhouse 
2018/03/10 16:19:08 AUTH: Starting
2018/03/10 16:19:09 AUTH: PASSED (whitelist) domain="domain1" address="127.0.0.1"
2018/03/10 16:19:09 AUTH: ALLOWED (CURVE) domain="domain1" client_key="(/ZgH}vi[e1L#W&{FSa{Lx:$L^$ijV888)BA+msr"
2018/03/10 16:19:09 AUTH: Stopping
2018/03/10 16:19:09 AUTH: Quitting: received QUIT message
2018/03/10 16:19:09 AUTH: Quit
2018/03/10 16:19:09 AUTH: Stopped
Ironhouse test OK
[docker:xenial] root:~/go/src/github.com/pebbe/zmq4/examples_security# ldd ./ironhouse
        linux-vdso.so.1 =>  (0x00007ffca375e000)
        libzmq.so.5 => /usr/lib/x86_64-linux-gnu/libzmq.so.5 (0x00007fa0d9101000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa0d8ee4000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa0d8b1a000)
        libsodium.so.18 => /usr/lib/x86_64-linux-gnu/libsodium.so.18 (0x00007fa0d88bc000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fa0d86b4000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fa0d8332000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa0d811c000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fa0d9367000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa0d7e13000)

Perhaps you need to install the package libsodium-dev first.

pebbe commented 6 years ago

Ah, I see you have libsodium-dev installed

omani commented 6 years ago

yes. I had installed libsodium already. but the error occured because when I switched to ubuntu packages I still had this one header file located in /usr/local/include stating version 4.2.4. and I guess version 4.2.4 was then incompatible with the libsodium version I already installed from the ubuntu repos for testing. so it was this one zmq.h in /usr/local/include messing up everything.

If libzmq.so.5 and zmq.h are not for the same version, you get in trouble.

so basically this happened, yes.

By the way, when I build a curve application on Xenial, it works fine

this is an important information to me. so maybe I don't need v4.2 at all to run my application, which would make things a little bit more easier. for example, distribution of binaries or deployments, since I am not depended on the opensuse repository anymore. thanks for that.

I am happy that everything works now.

Robhuman commented 4 years ago

~$ grep VERSION /usr/include/zmq.h

define ZMQ_VERSION_MAJOR 4

define ZMQ_VERSION_MINOR 2

define ZMQ_VERSION_PATCH 5

define ZMQ_MAKE_VERSION(major, minor, patch) \

define ZMQ_VERSION \

ZMQ_MAKE_VERSION (ZMQ_VERSION_MAJOR, ZMQ_VERSION_MINOR, ZMQ_VERSION_PATCH)

define ZMQ_PROTOCOL_ERROR_ZAP_BAD_VERSION 0x20000003

~$ grep VERSION /usr/local/include/zmq.h

define ZMQ_VERSION_MAJOR 4

define ZMQ_VERSION_MINOR 2

define ZMQ_VERSION_PATCH 5

define ZMQ_MAKE_VERSION(major, minor, patch) \

define ZMQ_VERSION \

ZMQ_MAKE_VERSION (ZMQ_VERSION_MAJOR, ZMQ_VERSION_MINOR, ZMQ_VERSION_PATCH)

define ZMQ_PROTOCOL_ERROR_ZAP_BAD_VERSION 0x20000003

zmq4 was installed with ZeroMQ version 4.1.4, but the application links with version 4.2.5