macOS: ts-node unable to load image from /opt/mqm/lib64

[mattgills]

Environment: macOS with MQ Toolkit Installed V 9.2.3.0 ibmmq: 0.9.18

I was able to create simple Node.js scripts that were based on the samples amqsget.js and amqsput.js. When running them with the command DYLD_LIBRARY_PATH=/opt/mqm/lib64 node <script_name>, I was able to successfully connect to and put a message to a remote MQ.

After establishing this base functionality, I wanted to utilize this package in an existing TypeScript project. I configured the start command to be ts-node <script_name> with the environment variable DYLD_LIBRARY_PATH=/opt/mqm/lib64 passed in using the dotenv package. When I attempted to start the application, the startup failed with error:

Cannot find MQ C library.
  Has the C client been installed?
  Have you set DYLD_LIBRARY_PATH?

/...path_to_my_project/node_modules/ffi-napi/lib/dynamic_library.js:75
    throw new Error('Dynamic Linking Error: ' + err);
          ^
Error: Dynamic Linking Error: dlopen(/opt/mqm/lib/libmqm_r.dylib, 2): image not found

If I build/compile the TypeScript into JavaScript, then I am able to run the compiled JavaScript with the node command successfully. Unfortunately the build process is not particularly quick, as there are other compilations in addition to TypeScript. I would like to utilize ts-node, if possible.

Troubleshooting Attempted

1. I installed ts-node globally to see if node had more permissions than ts-node, however that yielded the same outcome.
2. I added some additional logging statements in node_modules/ibmmq/mqi.js to see where node was succeeding and where ts-node was failing. It turns out that the node commands were were finding the C library using the redist directory in node_modules. However that directory does not exist, so I'm not sure how that is occurring. The ts-node commands search the redist directory and each of the /opt/mqm/... directories unsuccessfully.

I'm not quite sure if this library was only intended to work with node, let me know. Thanks.

[ibmmqmet]

I've never done anything before today with typescript, but I was able to run "ts-node amqsget" successfully from the sample directory so the basic mechanism would appear to work. At least on Linux,

You can set the environment variable MQIJS_DEBUG=true to have the code print out where it's trying to load libmqm from. But one thing I spotted in the dotenv doc was

We will never modify any environment variables that have already been set.

which makes me wonder whether you really have set the environment you think you have. Especially with the error message referring to "/opt/mqm/lib" instead of "lib64". Even when libmqm_r is located, for it to be succesfully loaded, its dependents like libmqmcs_r also have to be found by the loader.

[mattgills]

I was confused by the error message displaying "opt/mqm/lib" as well, however, the loadLibMulti function returns the last error message it encounters. Since "/opt/mqm/lib" is the last directory that is checked, that's what ends up displaying.

After setting MQIJS_DEBUG=true I see the same pattern. Node.js loads the library from the redist directory (that does not exist - see screenshot below) and ts-node fails to load the library from any directory.

Node.js

[ibmmq] 2021-10-18T15:03:36.970Z  : MQ library loaded from /...path-to-project/ibm-mq-testing/node_modules/ibmmq/redist/lib64 libmqm_r

ts-node

[ibmmq] 2021-10-18T15:02:25.511Z  : MQ library failed to be loaded from /...path-to-project/node_modules/ibmmq/redist/lib64 libmqm_r. Error Dynamic Linking Error: dlopen(/...path-to-project/node_modules/ibmmq/redist/lib64/libmqm_r.dylib, 2): image not found
[ibmmq] 2021-10-18T15:02:25.514Z  : MQ library failed to be loaded from <<default path>> libmqm_r. Error Dynamic Linking Error: dlopen(libmqm_r.dylib, 2): image not found
[ibmmq] 2021-10-18T15:02:25.530Z  : MQ library failed to be loaded from /opt/mqm/lib64 libmqm_r. Error Dynamic Linking Error: dlopen(/opt/mqm/lib64/libmqm_r.dylib, 2): Library not loaded: @rpath/libmqe_r.dylib
  Referenced from: /opt/mqm/lib64/libmqm_r.dylib
  Reason: image not found
[ibmmq] 2021-10-18T15:02:25.531Z  : MQ library failed to be loaded from /opt/mqm/lib64/compat libmqm_r. Error Dynamic Linking Error: dlopen(/opt/mqm/lib64/compat/libmqm_r.dylib, 2): image not found
[ibmmq] 2021-10-18T15:02:25.532Z  : MQ library failed to be loaded from /opt/mqm/lib libmqm_r. Error Dynamic Linking Error: dlopen(/opt/mqm/lib/libmqm_r.dylib, 2): image not found

[mattgills]

Also for more environment context: Node.js: 14.17.0 ts-node: 9.0.0

What versions do you run with? I feel like this would be a good place to check since you successfully ran with ts-node.

[mattgills]

It appears that there are few existing stack overflow discussions related to DYLD_LIBRARY_PATH:

• General Discussion
• macOS blocks DYLD_LIBRARY_PATH for sub-shells

I believe the second article is the root cause of the TypeScript process failing on Mac. I'm not sure what the solution is yet.

Short term workaround:

DYLD_LIBRARY_PATH=/opt/mqm/lib64 node ./node_modules/ts-node/dist/bin.js <ts file name>

[ibmmqmet]

1) The debug showing the library being loaded from the redist directory seems to be because MacOS - unforgivably in my opinion - will search the DYLD_LIBRARY_PATH EVEN WHEN an explicit path is given to dlopen(). So the library is being found, but not where mqi.js is telling the OS to look. Setting MQIJS_PREFER_INSTALLED_LIBRARY=true changes the order of the search requests so it is not so misleading. Maybe I'll set that automatically if running on MacOS in a future update. 2) I was able to find a networked Mac to play with, and can see the same problem with ts-node running the amqsput.js sample. The top-level library, _libmqmr, does get loaded but the next-level dependent _libmqer cannot be found which causes the step to fail. I've tried a variety of solutions, including explicitly setting DYLD_LIBRARY_PATH from within both the main program and mqi.js but it doesn't seem to get picked up. The only way I've found to resolve it so far requires a change to the node executable itself.

$ install_name_tool -add_rpath /opt/mqm/lib64 `which node`

That might not be an ideal solution, though it's not likely to harm any other Node programs, and I'll try to find something better, but it does seem to allow ts-node to run. It does at least hint at the problem area. I wonder if something else ought to get baked into the MQ client libraries - I'll drop a note to the people who do the builds for that platform.

[ibmmqmet]

sudo install_name_tool -add_rpath @loader_path /opt/mqm/lib64/libmqm_r.dylib

seems to work on my machine. And it still might not be ideal but it's a lot better than modifying the node binary idea.

[mattgills]

Thanks, that worked for me! I was having trouble finding the write arguments needed for that command, but the one you posted worked perfectly. I no longer need to specify the DYLD_LIBRARY_PATH and can run the app using ts-node.

[ibmmqmet]

for info: I've put a change into the product build/packaging processes that create the MQ MacOS toolkit. It adds the rpath element to dylib libraries so it won't need to be done manually once that version is available. I wasn't able to do it in time for this week's 9.2.4 release but it should show up in whatever the next version is in a few months.