ruby gem should contain libzmq

[kzk]

Currently, libzmq must be installed separately from gem.

For the ease of the installation, the gem itself should contain the c library itself (like cool.io contains libev etc). Also we cannot use libzmq on the PaaS platform like Heroku.

• Kazuki

[kenn]

+1

This is a major drawback in comparison with the hub-and-spoke broker model like RabbitMQ, where the broker setup overhead is just once and clients only need eventmachine + amqp gem. I think brokerless model really shines when installation is a snap on every machine.

[kybu]

Just popping in with my warped sense of humour saying that it would be nice if Ruby could install an operating system for me.

But seriously, libzmq is a system level wide library which should be managed by your OS. Installing your own version of software like this is a nightmare of every system administrator. Imagine a situation, where you've installed newer version of libzmq with a ruby gem and some software shipped by your Linux distribution uses the one bundled with distribution, obviously. If that new library happens to be in dynamic library load path, that software crashes.

While it is convenient, it is also insensible to the environment it runs on. Unfortunately, that seems to be the trend in Ruby land.

(I am not connected to zeromq project, just a happy user.)

[regularfry]

I'll just give this a quick +1. Relying on a packaged system library makes sense when the library is stable and the interface you're coding against will not change for the next few years. This is not the case for 0mq. If the attraction of keeping the gem simple is too high, a rbzmq-bundled fork would be appreciated.

[darix]

-1 intree libraries are the worst thing you can do. please stop being lazy and install the library system wide yourself if the OS doesnt come with it.

[regularfry]

Fortunately some of us are not so arrogant as to assume that we can predict everyone else's use cases. I would like to point those who are interested in this functionality in the direction of https://github.com/methodmissing/rbczmq. A mainstream, supported fork would still be appreciated.

[kybu]

Thanks for pointing out your wrapper. I will have a look if it suits to be used for my project. As for use cases, one size does not fit all, does it? ;)

[regularfry]

Not mine, just one I have found on my travels :-)

[darix]

This has nothing to do with arrogance. why should every of the bindings ship the library, if you can install it once and be done?

Let's say the library has a bug, then you would need to rebuild each binding you have installed, because every came with their intree copy. Instead of just fixing the one copy of the library on your system. in a normal case you would not even have to rebuild the bindings as the ABI/API remains the same.

Creating a proper package of the library for your distro/os, is usually a very simple task. often you can find already older versions of the package and can go from there.

[regularfry]

This has nothing to do with arrogance.

The arrogance isn't in what you're saying, it's in your assumption that anybody with a different viewpoint must self-evidently be wrong and hence deficient for disagreeing with you.

why should every of the bindings ship the library, if you can install it once and be done?

Because sometimes you can't install it once and be done. That's the mistake you're making. Gems give us a very handy way of switching between versions that are all installed at the same time. If I want to test against more than one version of zeromq, it's an utter pain to manage separate system-installed library versions. Rubygems already has a mechanism for precisely that purpose, and it makes no sense to duplicate it.

Let's say the library has a bug, then you would need to rebuild each binding you have installed, because every came with their intree copy. Instead of just fixing the one copy of the library on your system. in a normal case you would not even have to rebuild the bindings as the ABI/API remains the same.

I already know I'm likely to have more than one version; I specifically want old, buggy versions so I can check how my code will behave against distro-supplied packaged versions. This whole argument is moot.

It would also be really, really nice to have ruby library versions tied to native library versions to avoid the possibility of cross-version incompatibility.

Besides which, it's a net win to only have a single code management system. If that's laziness, it's the good sort.

[kybu]

regularfry, ok, I think I understand you, but let's do things properly then. libzmq depends on libpmg(multicast protocol) and in 2011, there were 7 releases. So, are you going to test every combination? I hope you get my point. That's why we have Linux distros and testing should be done by each distro and not every possible combination of software packages. And even better, is gem able to handle this kind of dependency well? If so, does it implement the same complicated logic as rpm, apt do?

The situation here is, that Ruby land evolves and changes rapidly. Fair enough, so fair enough that Debian and Ubuntu(not sure about others) don't let you install system wide gems, because it messes the whole point of having a stable Linux distribution.

I don't think that there is any way out at the moment. I just wish that Ruby folks who want to include external libraries in their gems spend some time administrating a Linux server, so they get the idea what is it about.

[regularfry]

By using rubygems at all, we've already opted out of the platform dependency management system. I'm not suggesting that the current library go away, or that you change the way you administer your servers. I'm suggesting that we be able to choose at what level the opt-out kicks in at. I'm suggesting that it be possible to choose to install libzmq with the gem so that when someone wants to do something with zmq which doesn't fit the typical server-sysadmin mould, or they're on a platform which is slow-moving, they don't end up replicating a whole lot of effort unnecessarily, and so that we can minimise the number of moving parts involved in provisioning our software.

I just wish that Ruby folks who want to include external libraries in their gems spend some time administrating a Linux server, so they get the idea what is it about.

Is your suggestion that I haven't? Because that would be comical - I work for a hosting company.

I am not making this suggestion from a position of ignorance; here we have an internal system for converting gems into debs which we absolutely rely on for day-to-day work. Rolling Debian packages for libraries is not something I find particularly overwhelming. And yet, here I am, supporting this feature.

[kybu]

I just wish that Ruby folks who want to include external libraries in their gems spend some time administrating a Linux server, so they get the idea what is it about.

Is your suggestion that I haven't? Because that would be comical - I work for a hosting company.

No need to be touchy. It's just my general observation that many Ruby folks are not that Unix skilled, hence for example basic shell commands are explained in RVM documentation.

And another problem has come to my mind. What if two gems include the libzmq, but each of a different version? What is going to happen if one tries to use both gems in the same process? You quite likely get the segfault. Also, you avoided answering the question how to handle additional dependencies of an included library. Libzmq is not that dependant, but others might well be. I am trying to say that including 3rd party libraries in gems is something which does more harm than is useful for reasons I have already stated.

P.S.: Converting gem to .deb sounds like a handy solution for some cases.

[regularfry]

And another problem has come to my mind. What if two gems include the libzmq, but each of a different version? What is going to happen if one tries to use both gems in the same process?

That would, indeed, be a problem. I can't imagine a situation where it would happen, though. For a start, different versions of the same gem can't collide. That's the point.

Also, you avoided answering the question how to handle additional dependencies of an included library.

In zmq's case, I can't see why including libpgm as well would be an issue. Then again, we have the example of Nokogiri which includes an extension which depends on a platform-supplied libxml2. At some point you have to draw the line and say "the platform needs to supply this" - what I and others are saying is that in zmq's case, the line is drawn inconveniently high. I want to be able to use zmq in the same fashion that I can use sqlite with amalgalite, and that's not officially supported right now.

[darix]

regarding the crash problem: it can happen if 2 different gems bring the same library intree in slightly different versions.

imagine nokogiri would bring libxml2 intree and a 2nd gem you use in your app pulls the libxml ruby bindings, which also have libxml2 intree in a slightly different version. crash. if you build them both against the libxml on your system it would just work. While you are right it wouldnt be such a big issue of 2 versions of the same gem. this scenario can still happen. you could work around it with an approach like the libv8 gem. so both the nokogiri and the libxml-ruby gem would depend on a libxml2 gem. but that is just as sick as the intree approach.

In any case ... if an upstream decides to offer intree libraries for easy deployment. they should have code that prefers the system copy and spits out a big warning if it uses the intree copy.

[rossmeissl]

Drama aside, it's a bummer that we can't use this gem on Heroku, where, of course, we don't have the ability to decide what libraries are installed at the system level. Has anybody had any further thoughts on this?

[seamusabshere]

hey @bwbuchanan do you have the resources you need to do what @kzk is suggesting?