Error: npm install iot-gateway-mqtt-http --save

[girishkumarbk]

I get this error when running this command. Any help ....

root@scson0016991001:~/cb/tmp/azure-iot-gateway-sdk/tools# npm install iot-gateway-mqtt-http --save npm http GET https://registry.npmjs.org/iot-gateway-mqtt-http npm http 404 https://registry.npmjs.org/iot-gateway-mqtt-http npm ERR! TypeError: Cannot read property 'latest' of undefined npm ERR! at next (/usr/share/npm/lib/cache.js:687:35) npm ERR! at /usr/share/npm/lib/cache.js:675:5 npm ERR! at saved (/usr/share/npm/node_modules/npm-registry-client/lib/get.js:142:7) npm ERR! at /usr/lib/nodejs/graceful-fs/polyfills.js:133:7 npm ERR! at Object.oncomplete (fs.js:107:15) npm ERR! If you need help, you may report this log at: npm ERR! http://github.com/isaacs/npm/issues npm ERR! or email it to: npm ERR! npm-@googlegroups.com

npm ERR! System Linux 3.13.0-32-generic npm ERR! command "/usr/bin/nodejs" "/usr/bin/npm" "install" "iot-gateway-mqtt-http" "--save" npm ERR! cwd /root/cb/tmp/azure-iot-gateway-sdk/tools npm ERR! node -v v0.10.25 npm ERR! npm -v 1.3.10 npm ERR! type non_object_property_load npm ERR! npm ERR! Additional logging details can be found in: npm ERR! /root/cb/tmp/azure-iot-gateway-sdk/tools/npm-debug.log npm ERR! not ok code 0

[damonbarry]

@girishkumarbk I took a look at this since you referred to it in the iot-edge repo. I don't have experience with these sample modules, but maybe I can at least help you get the gateway started without errors.

The instructions in this repo's readme.md are a little vague about the directories from which you do the various operations. Here's what I did (I won't be extremely detailed either, but hopefully this helps):

git clone https://github.com/Azure-Samples/iot-gateway-mqtt-http.git
cd iot-gateway-mqtt-http
npm install
npm pack  # produces iot-gateway-mqtt-http.tgz
cd ..

git clone https://github.com/Azure/iot-edge.git
cd iot-edge
v1/tools/build_nodejs.sh  # takes forever
export NODE_INCLUDE="path/to/iot-edge/v1/build_nodejs/dist/inc"
export NODE_LIB="path/to/iot-edge/v1/build_nodejs/dist/lib"
v1/tools/build.sh --enable-nodejs-binding --disable-native-remote-modules --disable-ble-module

cd v1/samples/nodejs_simple_sample/nodejs_modules
npm install #if you're going to use iothub_writer.js (see below)

cd ../../../build/samples/nodejs_simple_sample
npm install path/to/iot-gateway-mqtt-http/iot-gateway-mqtt-http.tgz

openssl genrsa -out tls-key.pem 2048
openssl req -new -sha256 -key tls-key.pem -out tls-csr.pem
openssl x509 -req -in tls-csr.pem -signkey tls-key.pem -out tls-cert.pem

This puts all the needed binaries, node.js packages, and certs into the same directory.

Now you can edit iot-edge/samples/nodejs_simple_sample/src/gateway_sample_lin.json. Remove all the existing entries from the "modules" array except for "iothub_writer" (or you could copy over the "iothub" and "mapper" module entries from iot-edge/v1/samples/simulated_device_cloud_upload/src/simulated_device_cloud_upload_lin.json--that's what I did), and add in the "mqtt" and "http" module entries as described in this repo's readme.md. You'll also need to replace the "links" array with the values from the readme.md. Finally, you'll need to add any connection string information (or device ID/key for "iothub"). Then you'd run it like this:

./nodejs_simple_sample ../../../samples/nodejs_simple_sample/src/gateway_sample_lin.json

I was able to run the sample. I didn't try connecting to either of the servers; I'll leave that to you. ;)

[girishkumarbk]

I seem to get this error while starting the nodejs_simple_sample ...

./nodejs_simple_sample ../../../samples/nodejs_simple_sample/src/gateway_sample_lin.json ERROR: Error: Cannot find module 'azure-iot-device-amqp'

[girishkumarbk]

Also is "iothub" module different from "iothubwriter" ?

[damonbarry]

@girishkumarbk The "iothub" module is written in C, and is the official module used in IoT Edge v1 for sending messages to IoT Hub. "iothub_writer" is more of a basic sample written in JavaScript. If you're just trying to get something going for the first time, either will work and iothub_writer might be easier to use. But if this turns into a long-term project you'd probably want to move to the iothub module.

RE the error, that's a missing Node.js dependency (azure-iot-device-amqp) in iothub_writer. I updated my instructions above, but basically you need to:

cd v1/samples/nodejs_simple_sample/nodejs_modules
npm install

[girishkumarbk]

I encountered ERROR: Error: Cannot find module 'memdown'

I resolved the same by updating the dependency in the package.json at node_modules/iot-gateway-mqtt-http/package.json: "memdown":"^2.0.0"

Able to run the sample with updated gateway_sample_lin.json to have mqtt->logger->iothub flow.. ./nodejs_simple_sample ../../../samples/nodejs_simple_sample/src/gateway_sample_lin.json

However, I see that when I push the messages into the mqtt broker via simple python mqtt clients (import paho.mqtt.client as mqtt) the message are not appearing in mqtt broker nor I'm able to see in the node that has subscribed for the topic of interest. Most of the time the broker is "segment faulting" in libnodejs library.

Couple of questions 1.0 Isn't the broker pass on the message to iothub or say logger in this case, if the links are provided in .json. Meaning mqtt->logger->iothub ? how to we achieve this ? 2.0 Does the mqtt broker republish the message to iotedge broker so that it could be caught by other modules like logger and iothub for processing.

Any thing missing ? please comment

[damonbarry]

1. The logger is optional. The links should be something like:
   • source: *, sink: logger
   • source: mqtt, sink: mapper
   • source: mapper, sink: iothub
   • source: iothub, sink: mapper
   • source: mapper, sink: mqtt

So there are two pipelines: (1) all messages go to the logger, and (2) messages flow from mqtt to mapper to iothub and (optionally) back.

2. Yes, the broker should pass on the messages but you need the mapper module.

If you’re seeing a segfault, you should post the output of running the app so we can see it.

[girishkumarbk]

I made the links changes as noted above. Still I see no messages in broker and it segfaults. Here is one such ..

MQTT Broker running.

Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffec353700 (LWP 19643)] 0x00007fffeea9001e in v8::internal::RememberedSet<(v8::internal::PointerDirection)1>::ClearInvalidSlots(v8::internal::Heap*) () from ../../bindings/nodejs/libnodejs_binding.so (gdb) bt

0 0x00007fffeea9001e in v8::internal::RememberedSet<(v8::internal::PointerDirection)1>::ClearInvalidSlots(v8::internal::Heap*) ()

from ../../bindings/nodejs/libnodejs_binding.so

1 0x00007fffeea6f7a4 in v8::internal::MarkCompactCollector::ClearInvalidRememberedSetSlots() () from ../../bindings/nodejs/libnodejs_binding.so

2 0x00007fffeea7913d in v8::internal::MarkCompactCollector::ClearNonLiveReferences() () from ../../bindings/nodejs/libnodejs_binding.so

3 0x00007fffeea7f0e9 in v8::internal::MarkCompactCollector::CollectGarbage() () from ../../bindings/nodejs/libnodejs_binding.so

4 0x00007fffeea5d6f0 in v8::internal::Heap::MarkCompact() () from ../../bindings/nodejs/libnodejs_binding.so

5 0x00007fffeea5dd9b in v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) ()

from ../../bindings/nodejs/libnodejs_binding.so

6 0x00007fffeea5e21f in v8::internal::Heap::CollectGarbage(v8::internal::GarbageCollector, char const, char const, v8::GCCallbackFlags) ()

from ../../bindings/nodejs/libnodejs_binding.so

7 0x00007fffeea5e596 in v8::internal::Heap::CollectAllGarbage(int, char const*, v8::GCCallbackFlags) ()

from ../../bindings/nodejs/libnodejs_binding.so

8 0x00007fffeea60698 in v8::internal::IncrementalMarkingJob::DelayedTask::RunInternal() () from ../../bindings/nodejs/libnodejs_binding.so

9 0x00007fffecee7e6c in v8::platform::DefaultPlatform::PumpMessageLoop(v8::Isolate*) ()

from /root/exp/iot-edge/v1/build_nodejs/dist/lib/libnode.so.48

10 0x00007fffece6d167 in node::Start(int, char**) () from /root/exp/iot-edge/v1/build_nodejs/dist/lib/libnode.so.48

11 0x00007fffee8141ff in nodejs_module::ModulesManager::NodeJSRunner (this=0x5555557ae340)

at /root/exp/iot-edge/v1/bindings/nodejs/src/modules_manager.cpp:259

12 0x00007fffee814293 in nodejs_module::ModulesManager::NodeJSRunnerInternal (user_data=0x5555557ae340)

at /root/exp/iot-edge/v1/bindings/nodejs/src/modules_manager.cpp:272

13 0x00007ffff7bcd8c6 in ThreadWrapper (threadInstanceArg=0x5555557adc80) at /root/exp/iot-edge/v1/deps/c-utility/adapters/threadapi_pthreads.c:34

14 0x00007ffff735b184 in start_thread (arg=0x7fffec353700) at pthread_create.c:312

15 0x00007ffff708803d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

(gdb)

[girishkumarbk]

Second .. seg fault

MQTT Broker running. Error in `/root/exp/iot-edge/v1/build/samples/nodejs_simple_sample/nodejs_simple_sample': double free or corruption (out): 0x00007fffe40f5990

Program received signal SIGABRT, Aborted. [Switching to Thread 0x7fffec353700 (LWP 19761)] 0x00007ffff6fc0c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt

0 0x00007ffff6fc0c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56

1 0x00007ffff6fc4028 in __GI_abort () at abort.c:89

2 0x00007ffff6ffd2a4 in __libc_message (do_abort=do_abort@entry=1, fmt=fmt@entry=0x7ffff710f350 " Error in `%s': %s: 0x%s \n")

at ../sysdeps/posix/libc_fatal.c:175

3 0x00007ffff700982e in malloc_printerr (ptr=, str=0x7ffff710f480 "double free or corruption (out)", action=1) at malloc.c:4998

4 _int_free (av=, p=, have_lock=0) at malloc.c:3842

5 0x00007ffff75d0fcd in CRYPTO_free () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0

6 0x00007ffff761c606 in ?? () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0

7 0x00007ffff761b2e7 in EC_POINT_free () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0

8 0x00007ffff761b33f in EC_GROUP_free () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0

9 0x00007ffff7623d1e in EC_KEY_free () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0

10 0x00007ffff798818a in ?? () from /lib/x86_64-linux-gnu/libssl.so.1.0.0

11 0x00007ffff798741e in SSL_free () from /lib/x86_64-linux-gnu/libssl.so.1.0.0

12 0x00007fffecede84a in node::crypto::SSLWrap::DestroySSL() () from /root/exp/iot-edge/v1/build_nodejs/dist/lib/libnode.so.48

13 0x00007fffecee3afa in node::TLSWrap::DestroySSL(v8::FunctionCallbackInfo const&) ()

from /root/exp/iot-edge/v1/build_nodejs/dist/lib/libnode.so.48

14 0x00007fffeecf0b17 in v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo const&)) ()

from ../../bindings/nodejs/libnodejs_binding.so

15 0x00007fffee89aeb5 in v8::internal::(anonymous namespace)::HandleApiCallHelper(v8::internal::Isolate*, v8::internal::(anonymous namespace)::BuiltinArguments<(v8::internal::BuiltinExtraArguments)3>) () from ../../bindings/nodejs/libnodejs_binding.so

16 0x00007fffee89b74f in v8::internal::Builtin_HandleApiCall(int, v8::internal::Object*, v8::internal::Isolate) ()

from ../../bindings/nodejs/libnodejs_binding.so

17 0x00003d26024060c7 in ?? ()

18 0x00003d2602406001 in ?? ()

19 0x00007fffec352520 in ?? ()

20 0x0000000300000000 in ?? ()

21 0x00007fffec352578 in ?? ()

22 0x00003d26027bded7 in ?? ()

---Type <retu

[damonbarry]

Are you building Node.js by calling v1/tools/build_nodejs.sh without any arguments? Or are you doing something different (like passing the --node-version argument, or building Node.js a different way)?

Also, you provided two different call stacks: the segfault and the double free. Are these happening at the same time in the same run, or did they happen separately during different runs? Did they happen as soon as you started the app, or after you tried to connect an MQTT client, or when the client sent its first message, or something else?

[girishkumarbk]

Are you building Node.js by calling v1/tools/build_nodejs.sh without any arguments?

Yes. As indicated in the instructions given above.

The segfaults happen at different times in different runs. Some times I see segfault at the first mqtt connection and sometimes after few attempts to connection and publishing data is made. The segfaults are not consistent.

Thanks.

[damonbarry]

OK, at this point it sounds like you're running into problems with the sample itself rather than the Azure IoT Edge v1 libraries (not ruling it out, of course, but that's what it looks like to me). I don't think I can offer much more assistance, but the sample's maintainers might have more information for you.

[girishkumarbk]

Ok. Thanks for your support on this issue. Wondering ... 1.0) If the Azure mqtt sample maintainers are looking into issues in this repositories ? 2.0) I still feel v1 edge main repository should have had the mqtt/http module with it.

[girishkumarbk]

Do you see any reason why the git repository URL is mentioned differently in the repository README

git clone --recursive https://github.com/Azure/azure-iot-gateway-sdk.git git checkout 287beed07490d98a24a4e9ddd33ec7127fc3acbf

The URL is different (Its not pointing to Azure samples) Does this work only particular checkout "287beed07490d98a24a4e9ddd33ec7127fc3acbf" ?

[damonbarry]

The iot-edge repo used to be named azure-iot-gateway-sdk. And when you rename a GitHub repo, the old name continues to work.