brianc / node-libpq

Simple, low level native bindings to PostgreSQL's libpq from node.js
111 stars 41 forks source link

nodejs process die with stacktrace when using SSL #27

Open axvpast opened 9 years ago

axvpast commented 9 years ago

Used libpq -> pg-native -> sequelize -> our source code Used connection to PostgreSQL with mandatory SSL (openssl). sequelize configured with connection pool with values like { min: 15, max: 15 } In ~10% restarts in first second we got stacktrace like:

Connection parameters: user='KEX' password='test' port=* glibc detected * node: double free or corruption (fasttop): 0x000000000208ca50 *** /lib64/libc.so.6(+0x7d4e6)[0x7fde9b88b4e6] /usr/lib64/libcrypto.so.1.0.0(CRYPTO_free+0x1d)[0x7fde9cb44f2d] /usr/lib64/libcrypto.so.1.0.0(OBJ_NAME_add+0xa0)[0x7fde9ca5d500] /usr/lib64/libcrypto.so.1.0.0(EVP_add_digest+0x56)[0x7fde9cae22e6] /usr/lib64/libssl.so.1.0.0(SSL_library_init+0xee)[0x7fde9ce189de] /usr/lib64/libpq.so.5(+0x1b65c)[0x7fde9b3ee65c] /usr/lib64/libpq.so.5(PQconnectPoll+0xe82)[0x7fde9b3e0492] /usr/lib64/libpq.so.5(+0xdb8e)[0x7fde9b3e0b8e] /usr/lib64/libpq.so.5(PQconnectdb+0x1f)[0x7fde9b3e151f] /xxx/psql-init-stacktrace/node_modules/libpq/build/Release/addon.node(_ZN10Connection9ConnectDBEPKc+0x2a)[0x7fde9b604e0a] /xxx/psql-init-stacktrace/node_modules/libpq/build/Release/addon.node(_ZN18ConnectAsyncWorker7ExecuteEv+0x35)[0x7fde9b609645] node[0x5c245d]

According enabled logs from node-libpq sequelize open 15 connections to server (in parallel) and some of this connection try to init SSL fails with stacktrace & process die.

OS Linux gentoo && debian - reproduce ~10% cases. Crash is not dependent from postgress client installation, and openssl/nodejs version.

MACOS 10.10.14 - cannot reproduce.

full nodejs module versions: ├─┬ concat-stream@1.5.0 │ ├── inherits@2.0.1 │ ├─┬ readable-stream@2.0.2 │ │ ├── core-util-is@1.0.1 │ │ ├── isarray@0.0.1 │ │ ├── process-nextick-args@1.0.2 │ │ ├── string_decoder@0.10.31 │ │ └── util-deprecate@1.0.1 │ └── typedarray@0.0.6 ├─┬ libpq@1.6.4 │ ├── bindings@1.2.1 │ └── nan@1.9.0 ├─┬ log4js@0.6.26 │ ├── async@0.2.10 │ ├─┬ readable-stream@1.0.33 │ │ ├── core-util-is@1.0.1 │ │ ├── inherits@2.0.1 │ │ ├── isarray@0.0.1 │ │ └── string_decoder@0.10.31 │ ├── semver@4.3.6 │ └── underscore@1.8.2 ├── moment@2.10.6 ├─┬ pg@4.4.1 │ ├── buffer-writer@1.0.0 │ ├── generic-pool@2.1.1 │ ├── packet-reader@0.2.0 │ ├── pg-connection-string@0.1.3 │ ├─┬ pg-types@1.10.0 │ │ ├── ap@0.2.0 │ │ ├── postgres-array@1.0.0 │ │ ├── postgres-bytea@1.0.0 │ │ ├── postgres-date@1.0.0 │ │ └─┬ postgres-interval@1.0.0 │ │ └── xtend@4.0.0 │ ├─┬ pgpass@0.0.3 │ │ └─┬ split@0.3.3 │ │ └── through@2.3.8 │ └── semver@4.3.6 ├── pg-copy-streams@0.3.0 ├── pg-cursor@1.0.0 ├── pg-hstore@2.3.2 ├─┬ pg-native@1.8.1 │ ├── pg-types@1.6.0 │ └─┬ readable-stream@1.0.31 │ ├── core-util-is@1.0.1 │ ├── inherits@2.0.1 │ ├── isarray@0.0.1 │ └── string_decoder@0.10.31 ├─┬ pg-query-stream@0.7.0 │ └─┬ readable-stream@1.1.13 │ ├── core-util-is@1.0.1 │ ├── inherits@2.0.1 │ ├── isarray@0.0.1 │ └── string_decoder@0.10.31 ├── q@1.4.1 ├─┬ sequelize@3.5.1 │ ├── bluebird@2.9.34 │ ├── depd@1.0.1 │ ├── dottie@0.3.1 │ ├── generic-pool@2.2.0 │ ├── inflection@1.7.1 │ ├── lodash@3.10.1 │ ├── moment-timezone@0.4.0 │ ├── node-uuid@1.4.3 │ ├── semver@5.0.1 │ ├── shimmer@1.0.0 │ ├── toposort-class@0.3.1 │ ├── validator@3.43.0 │ ├─┬ wellknown@0.3.1 │ │ ├─┬ concat-stream@1.4.10 │ │ │ ├── inherits@2.0.1 │ │ │ ├─┬ readable-stream@1.1.13 │ │ │ │ ├── core-util-is@1.0.1 │ │ │ │ ├── isarray@0.0.1 │ │ │ │ └── string_decoder@0.10.31 │ │ │ └── typedarray@0.0.6 │ │ └── minimist@0.0.2 │ └── wkx@0.0.7 └── underscore@1.8.3

axvpast commented 9 years ago

Possible solution - add to libpq/src/addon.cc

--- addon.cc.orig       2015-08-13 14:20:34.000000000 +0300
+++ addon.cc    2015-08-13 14:19:49.000000000 +0300
@@ -69,6 +69,8 @@
   NODE_SET_PROTOTYPE_METHOD(tpl, "$cancel", Connection::Cancel);

   exports->Set(NanNew<v8::String>("PQ"), tpl->GetFunction());
+
+  PQinitSSL( 0 );
 }

 NODE_MODULE(addon, InitAddon)

This code disable SSL initialization by native libpq and use already initialized SSL by nodejs.