Cannot find a way to build dependency dlls (libsodium and libzmq) into ZeroMQ.dll manifest

[websteroy]

I need to deploy libzmq.dll and libsodium.dll as part of ZeroMQ.dll's manifest. There's code in the Platform.Win32.cs source file that extracts these dependencies to AppData's temporary folder. However, I need to know if there is already a mechanism that is used to create a manifest for ZeroMQ.dll that would contain these dlls as part of the manifest. Right now, in code, I do not see any such thing.

In summary, my goal is to be able to deploy the ZeroMQ.dll, with manifest that contains the unmanaged dependencies which can be extracted at runtime. Do you have some ready made configuration that I could use?

@metadings Can you please help me here?

Thanks!

[metadings]

I don't get your question...

ZeroMQ does actually load the files from i386/amd64 folder, then it tries to load the files from system paths and finally it tries to extract a resource and load it. However there is no resource.

What do you mean by manifest?

[websteroy]

Hi Uli,

Yeah, my question is about the loading of the resource. Please ignore the previous message for now.

So far, what I have done is that I changed the "Build Action" property of the libzmq.dll and libsodium.dll to "Embedded Resource".

In method ExtractManifestResource, the resourceName variable contains a string like "libzmq.amd64.dll". But, shouldn't it be "ZeroMQ.amd64.libzmq.dll"?

The following is the existing code extract from file lib\Platform.Win32.cs:

string resourceName = string.Format(string.Format("{0}.{1}{2}", libraryName, arch, LibraryFileExtension));

libraryName = libzmq arch = amd64

I changed it to:

string resourceName = string.Format(string.Format("ZeroMQ.{0}.{1}{2}", arch, libraryName, LibraryFileExtension));

And, now it works like a charm.

As far as I understand, the resource is built into the assembly based on the folder structure used. So, ZeroMQ\amd64\libzmq.dll gets converted to ZeroMQ.amd64.libzmq.dll.

Please correct me if I'm wrong.

P.S. My goal is to deploy ZeroMQ.dll and its dependencies as a single dll. I do not want to place libzmq.dll and libsodium.dll separately.

[metadings]

Hey littlesnake!

I like very much, that you're using the embedded resource at compilation; that's how I did it back then.

However I don't like to say twice ZeroMQ and libzmq.dll/ libzmq.so. Can't you rename that resource, to be amd64.libzmq.dll for example?

[websteroy]

The embedded resources which are part of the assembly are accessed based on their location in the assembly. so, if I want to access and extract libzmq.dll which is in folder amd64, which in turn is in assembly "ZeroMQ", we should form the absolute string - "ZeroMQ.amd64.libzmq.dll". That is how they are referenced in the assembly. Current implementation forms the string - "libzmq.amd64.dll".

Here's a snapshot of the resources present in the assembly. You can see what they are named as.

I'm not using the term ZeroMQ, it is automatically used by the build to identify the resource correctly within the assembly.

[metadings]

hehe, you are really good in VC#... Would you like to make a pull request, to rename the resource files in the Posix and Win32 implementations? well wait...

Now you want to do a git pull origin master, the files Platform.Posix and .Win32 are now being updated to ZeroMQ.amd64.libzmq.dll/ZeroMQ.amd64.libzmq.so...

[websteroy]

Yes, I will do that. I can check for Win32, however checking for Posix would be difficult.

Meanwhile, a few more questions:

1. Any reason why clrzmq4 does not use the LGPL license used by the libzmq project? I assume you want to allow the binding to be used by commercial tools if they need it. Am I right?
2. This binding doesn't appear on the CLR bindings page. http://zeromq.org/bindings:clr Any reason why?

[metadings]

1. I dislike the idea of "a license" - which license should I use? Public Domain, without warrant.
2. No, I didn't actually change the CLR bindings page, would you like to update the clrzmq binding?

[websteroy]

1. I think you should use the same license used by the libzmq project - LGPL Or maybe you can even take Pieter's opinion on this.
2. I think the CLR bindings page should be updated to make this binding official. I see you already have Pieter's approval for the binding (http://grokbase.com/t/zeromq/zeromq-dev/1519tz7jz6/new-clrzmq-add-it-to-github). So, why not add it to the bindings page and make it official to the public?

[metadings]

How to change the bindings:clr page, do I need an editor account or just a git push ...? ;-)

There are some cool examples in the zguide, HWClient and HWServer, but also the Espresso...

Did you try them?

Updating... Please update your git repository, using git pull origin master... well now do it, this is the patch...

I need an update for the Platform.*.cs files... The program is written to load the defaults first (should we have default paths, like Environment variables?), then override by Platform (Win32 or Posix or Posix > MacOSX). There may also be someone who checks the installed runtime, and loads them by the boost naming scheme...

I am testing this on

• Windows 7 x86 Core 2 Duo Notebook with VC2010 and .NET 4.x, and
• ubuntu 14.04 amd64 Core 2 Quad Desktop with GNU C v4.8.2 compiler and mono 3.12.

[websteroy]

I'm new to this. So, even I don't know how the page should be updated.

[metadings]

Well, let's wait on Pieter's opinion on the licensing issue... I'm also glad to update the AssemblyInfo.cs and ZeroMQ.nuspec... How should I say in "Authors" and "Copyright"?

Please download clrzmq4 using git clone https://github.com/zeromq/clrzmq4 then you just need git pull origin master to get the current ZeroMQ CLR namespace on your computer...

[websteroy]

Uli, one more question.

The currrent repository contains libzmq version 4.1. Is it possible that after I fix the issue, we could release a build with the stable release 4.0.4?

I would prefer to work with a stable release instead of an RC.

[metadings]

zeromq4-1 is almost stable, but you can also use zeromq4-x. I don't want to move back to zeromq4-x.

Just replace the files in i386 and amd64.

The zmq.sizeof_zmq_msg_t is also changing with zeromq4-x to zeromq4-1, so you have to set

public static readonly int sizeof_zmq_msg_t = sizeof_zmq_msg_t_v3;

Now there is if (minor == 0) sizeof_zmq_msg_t = sizeof_zmq_msg_t_v3; - you just need to place the libzmq.dll/.so to the i386/amd64 folder.

I do not recommend switching back, looking on libzmq :-)

[hintjens]

For licensing I'd suggest using MPL v2 as we're doing in newer projects. There's a page on the community wiki that explains this. On Feb 18, 2015 11:13 AM, "Uli Riehm" notifications@github.com wrote:

Well, let's wait on Pieter's opinion on the licensing issue... I'm also glad to update the AssemblyInfo.cs and ZeroMQ.nuspec... How should I say in "Author" and "License"?

— Reply to this email directly or view it on GitHub https://github.com/zeromq/clrzmq4/issues/13#issuecomment-74841014.

[metadings]

I would like to say "Alle Angaben ohne Gewähr.", however Americans like to say "Alle Angaben ohne irgendeine Gewähr, explizit und inklusive ... "

I am using LGPL now.

[metadings]

Beware the v4.0.4 is the file in miru's archives, they are originally from 2014-03-15!

Current version of zeromq4-x is v4.0.5 (v4.0.6 the next one).

We need to update the Installers for Windows section on zeromq.org... Shouldn't we prefer zeromq4-1 instead of zeromq4-x now and use libzmq then?

[gillima]

Using a code like this would allow to add the libraries to any assembly of the Project. So a recompile of clrzmq4 would not be necessary.

            if (resourceStream == null)
            {
                // Locate the resource in any of the current loaded assemblies
                var resourceAssembly = AppDomain.CurrentDomain
                    .GetAssemblies()
                    .FirstOrDefault(ass => ass.GetManifestResourceNames().Contains(resourceName));

                if (resourceAssembly != null)
                    resourceStream = resourceAssembly.GetManifestResourceStream(resourceName);
            }

Also the name of a embedded resource can be defined for Visual Studio. It's not possible with the UI, but by editing the .csproj with a text editor the names can be defined like this:

 <ItemGroup>
    <EmbeddedResource Include="amd64\libsodium.dll">
      <LogicalName>ZeroMQ.libsodium.amd64.dll</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="amd64\libsodium.so">
      <LogicalName>ZeroMQ.libsodium.amd64.so</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="amd64\libzmq.dll">
      <LogicalName>ZeroMQ.libzmq.amd64.dll</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="amd64\libzmq.so">
      <LogicalName>ZeroMQ.libzmq.amd64.so</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="i386\libsodium.dll">
      <LogicalName>ZeroMQ.libsodium.i386.dll</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="i386\libsodium.so">
      <LogicalName>ZeroMQ.libsodium.i386.so</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="i386\libzmq.dll">
      <LogicalName>ZeroMQ.libzmq.i386.dll</LogicalName>
    </EmbeddedResource>
    <EmbeddedResource Include="i386\libzmq.so">
      <LogicalName>ZeroMQ.libzmq.i386.so</LogicalName>
    </EmbeddedResource>
  </ItemGroup>

[metadings]

Yes, that looks correctly... I decided not to have it as resource, because it will add some weight to the binary. You need to (re)compile ZeroMQ/clrzmq4 with your EmbeddedResources.

I find it very interesting, that you ask AppDomain.CurrentDomain.GetAssemblies() for having one of these resources. I didn't think about the possibilities out there :) But: There are serious security problems with loading just some byte[] from some assembly...

[metadings]

AppDomain.CurrentDomain.GetAssemblies() on a x86 Windows machine

[
    "Assembly" { "Location": "C:\Windows\Microsoft.NET\Framework\v4.0.30319\mscorlib.dll" },
    "Assembly" { "Location": "C:\Projects\YourProject\bin\Debug\YourProject.exe" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Core\v4.0_4.0.0.0__b77a5c561934e089\System.Core.dll" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll" },
    "Assembly" { "Location": "C:\Projects\YourProject\bin\Debug\ZeroMQ.dll" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Configuration\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Configuration.dll" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Xml\v4.0_4.0.0.0__b77a5c561934e089\System.Xml.dll" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Security\v4.0_4.0.0.0__b03f5f7f11d50a3a\System.Security.dll" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.Data.SqlXml\v4.0_4.0.0.0__b77a5c561934e089\System.Data.SqlXml.dll" },
    "Assembly" { "Location": "C:\Windows\Microsoft.Net\assembly\GAC_32\System.Data\v4.0_4.0.0.0__b77a5c561934e089\System.Data.dll" }
]

AppDomain.CurrentDomain.GetAssemblies() on an amd64 Linux machine

[
    "Assembly" { "Location": "/usr/lib/mono/4.5/mscorlib.dll" },
    "Assembly" { "Location": "/home/metadings/Projects/YourProject/bin/Debug/YourProject.exe" },
    "Assembly" { "Location": "/usr/lib/mono/gac/System.Core/4.0.0.0__b77a5c561934e089/System.Core.dll" },
    "Assembly" { "Location": "/home/metadings/Projects/YourProject/bin/Debug/ZeroMQ.dll" },
    "Assembly" { "Location": "/usr/lib/mono/gac/System/4.0.0.0__b77a5c561934e089/System.dll" },
    "Assembly" { "Location": "/usr/lib/mono/gac/System.Configuration/4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll" },
    "Assembly" { "Location": "/usr/lib/mono/gac/System.Xml/4.0.0.0__b77a5c561934e089/System.Xml.dll" },
    "Assembly" { "Location": "/home/metadings/Projects/YourProject/bin/Debug/System.Data.SQLite.dll" },
    "Assembly" { "Location": "/usr/lib/mono/gac/System.Data/4.0.0.0__b77a5c561934e089/System.Data.dll" },
    "Assembly" { "Location": "/usr/lib/mono/gac/Mono.Security/4.0.0.0__0738eb9f132ed756/Mono.Security.dll" }
]

[gillima]

Might be correct, but is it less secure than have them in subfolders of the application...

[metadings]

Hm... It's difficult to weight out flexibility and security... Don't you mean "more" secure? :) You are right, probably should I trust a compiled assembly more than some loosely cupled files.

I think there should be some configuration directive where you can tell, if you first want the libzmq.dll from C:\Windows\system32, or you first want the one from ~\amd64, or you're just trusting the version in the resources; a place where I also can say from which assemblies I want libzmq.* to be extracted; a place where I also can say to which temp folder I want the extracted assembly resource to be.

[gillima]

Sounds cool to me. Should not be to hard to achieve as long as the configuration is done before the first call into platform...

[metadings]

This is so awesome...! :) I need to copy over my HTTPDealerDevice... STREAM sockets do split the message by 8192 bytes; my HTTPDealerDevice actually does read the Content-Length from the header and then REQuests/REPlies the full message...

[asgerhallas]

Hi, I'm having some trouble using this in an ASP.NET environment where it seems to try to locate the asseblies from my user/AppData/Temp directory. After having read this thread, I'm actually not really sure what the conclusion is...

Is it possible to get the asseblies from embedded ressources in the current library (as it is on nuget)? Or is this a change coming? Or am I just out of luck? :)

[metadings]

OH, I didn't test this in an ASP.NET environment... (I'm running my own one, I don't know how to call it, it's just ZeroMQ, an HttpDealerDevice and a bit of CSharpCodeProvider ;))

You need to add the libraries as an embedded resource and recompile ZeroMQ/clrzmq4, instead of just adding them using "copy if newer". This is not done because I want to be flexible in using libzmq and I also don't want the file becoming too big, however you can do so.

[asgerhallas]

Ok, thank you for the quick response (and sorry for my late one). That'll work :) Would you consider maybe having to nuget packages, one as now and one with the files embedded? For easier access to this solution?

[metadings]

Yes, I plan to do so... I need new binaries, because I compiled zeromq4-1 a year ago. However, I don't have a Windows machine, so this is being delayed right now...

(I need a Windows binary using VC2010 (?), a Linux binary using GCC 4.8.4 (?) and maybe also MacOSX and Android binaries.)

[asgerhallas]

Fantastic!

Now I don't know how this works exactly, but isn't it the same binaries as those deployed on the ZeroMQ website via the Windows installer?

Or else I might be able to do the Windows build unless it's seriously complicated :)

[metadings]

Well it is a bit complicated... :-)

And NO, the binary deployed in the "Windows installer" on the website is seriously outdated, it dates back to 2014.

[asgerhallas]

Ok. I'll stand by then. Let me know if there's anything I should do! :)

[asgerhallas]

Hi again, I see there's a new release, cool! Does this mean there's a chance of a nuget-package with embedded binaries? :)

[miaf]

Hi there. Sorry I am very interested by the discussion but I am not getting you well. I am facing the same problem when trying to let my frontend asp.net core communicate with the backend and get the following error details InnerException: HResult=-2147024894 Message=UnmanagedLibrary[libzmq] Unable to load binary "libzmq" from folders C:\Works\Pro\HSMS\WebPortal\bin\Debug\net461\win7-x86\i386\libzmq.dll C:\Works\Pro\HSMS\WebPortal\bin\Debug\net461\win7-x86\x86\libzmq.dll C:\Windows\system32\libzmq.dll For simplicity, I just copy the i386 folder into C:\Works\Pro\HSMS\WebPortal\bin\Debug\net461\win7-x86\ and everything is working find. I think trying the others way work as well.

[metadings]

Closing this issue. It's too old