search

buddycloud / buddycloud-http-api

The buddycloud API server written in Node.js
https://buddycloud.org/wiki/Buddycloud_HTTP_API
39 stars 17 forks source link

server crashing when replying to a request #18

Closed highfellow closed 11 years ago

highfellow commented 11 years ago

Hi,

I've been trying to set up the http api server. I've built it on the webhost I'm using, and started it with npm start. I've been testing it by sending the following request:

[highfellow@web225 ~]$ curl -v --basic -u andy@highfellow.org:xxx localhost:22024/andy@highfellow.org/content/posts?max=1
* About to connect() to localhost port 22024
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 22024
* Server auth using Basic with user 'andy@highfellow.org'
> GET /andy@highfellow.org/content/posts?max=1 HTTP/1.1
> Authorization: Basic *(removed)*
> User-Agent: curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: localhost:22024
> Accept: */*
> 
* Empty reply from server
* Connection #0 to host localhost left intact
curl: (52) Empty reply from server
* Closing connection #0

The server contacts the xmpp server, which replies with a response, at which point the http server seems to crash:

[highfellow@web225 buddycloud-http-api]$ npm start

> buddycloud-http-api@0.0.1 start /home/highfellow/build/buddycloud-http-api
> node ./server.js

Cannot load StringPrep-0.1.0 bindings. You may need to `npm install node-stringprep'
Warning: program compiled against libxml 209 using older 206
Server started with configuration profile "production"
Listening on port 22024

OUT xmpp: <iq type="get" from="andy@highfellow.org/220069a8-cff7-488d-acbf-821833740bec" to="buddycloud.highfellow.org" id="f341a31022293a4cc62840bbc7cfe110"><pubsub xmlns="http://jabber.org/protocol/pubsub"><items node="/user/andy@highfellow.org/posts" max_items="1"/><set xmlns="http://jabber.org/protocol/rsm"><max>1</max></set></pubsub></iq>
IN xmpp: <iq id="f341a31022293a4cc62840bbc7cfe110" type="result" to="andy@highfellow.org/220069a8-cff7-488d-acbf-821833740bec" from="buddycloud.highfellow.org" xmlns:stream="http://etherx.jabber.org/streams"><pubsub xmlns="http://jabber.org/protocol/pubsub"><items node="/user/andy@highfellow.org/posts"><item id="4d92da42-aaa1-4374-80b5-dce5d5575809"><entry xmlns="http://www.w3.org/2005/Atom"><content>Glad you like it :-)</content><author><name>andy@highfellow.org</name><uri>acct:andy@highfellow.org</uri></author><in-reply-to xmlns="http://purl.org/syndication/thread/1.0" ref="6376aa0d-5d95-4fa1-a37f-976ac429113e"/><id>4d92da42-aaa1-4374-80b5-dce5d5575809</id><published>2012-10-13T00:51:01.442Z</published><updated>2012-10-13T00:51:01.442Z</updated><link rel="self" href="xmpp:buddycloud.highfellow.org?pubsub;action=retrieve;node=/user/andy@highfellow.org/posts;item=4d92da42-aaa1-4374-80b5-dce5d5575809"/><verb xmlns="http://activitystrea.ms/spec/1.0/">post</verb><object xmlns="http://activitystrea.ms/spec/1.0/"><object-type>comment</object-type></object></entry></item></items><set xmlns="http://jabber.org/protocol/rsm"><first index="0">4d92da42-aaa1-4374-80b5-dce5d5575809</first><last>4d92da42-aaa1-4374-80b5-dce5d5575809</last><count>590</count></set></pubsub></iq>
[highfellow@web225 buddycloud-http-api]$

The response from the xmpp server is never returned as an http request (see the first pasted section).

The only thing I remember doing differently when built the server was that I downgraded the version of node-stringprep to 0.1.4 to get it to build (with 0.1.5 it gave a build error).

If you have any ideas what is happening, or how to debug this further, that would be helpful.

Cheers,

andy@highfellow.org

highfellow commented 11 years ago

I've just had a go at debugging this with the nodejs debugger and node-inspector, but the server crashes at the point where I send -USR1 to the process.

[highfellow@web225 buddycloud-http-api]$ node --debug-brk=27947 server.js 
debugger listening on port 27947
User defined signal 1
[highfellow@web225 buddycloud-http-api]$
highfellow commented 11 years ago

If I start the server with 'node server.js' instead of 'npm start', it does pretty much the same thing when I make a request, but ends with a segfault message:

[highfellow@web225 buddycloud-http-api]$ node server.js 
Cannot load StringPrep-0.1.0 bindings. You may need to `npm install node-stringprep'
Warning: program compiled against libxml 209 using older 206
Server started with configuration profile "production"
Listening on port 22024
OUT xmpp: <iq type="get" from="andy@highfellow.org/97e83fed-4d0f-46ff-8aaf-03adb9587465" to="buddycloud.highfellow.org" id="1549e368252adf0b24718ee3f4d60835"><pubsub xmlns="http://jabber.org/protocol/pubsub"><items node="/user/andy@highfellow.org/posts" max_items="1"/><set xmlns="http://jabber.org/protocol/rsm"><max>1</max></set></pubsub></iq>
IN xmpp: <iq id="1549e368252adf0b24718ee3f4d60835" type="result" to="andy@highfellow.org/97e83fed-4d0f-46ff-8aaf-03adb9587465" from="buddycloud.highfellow.org" xmlns:stream="http://etherx.jabber.org/streams"><pubsub xmlns="http://jabber.org/protocol/pubsub"><items node="/user/andy@highfellow.org/posts"><item id="4d92da42-aaa1-4374-80b5-dce5d5575809"><entry xmlns="http://www.w3.org/2005/Atom"><content>Glad you like it :-)</content><author><name>andy@highfellow.org</name><uri>acct:andy@highfellow.org</uri></author><in-reply-to xmlns="http://purl.org/syndication/thread/1.0" ref="6376aa0d-5d95-4fa1-a37f-976ac429113e"/><id>4d92da42-aaa1-4374-80b5-dce5d5575809</id><published>2012-10-13T00:51:01.442Z</published><updated>2012-10-13T00:51:01.442Z</updated><link rel="self" href="xmpp:buddycloud.highfellow.org?pubsub;action=retrieve;node=/user/andy@highfellow.org/posts;item=4d92da42-aaa1-4374-80b5-dce5d5575809"/><verb xmlns="http://activitystrea.ms/spec/1.0/">post</verb><object xmlns="http://activitystrea.ms/spec/1.0/"><object-type>comment</object-type></object></entry></item></items><set xmlns="http://jabber.org/protocol/rsm"><first index="0">4d92da42-aaa1-4374-80b5-dce5d5575809</first><last>4d92da42-aaa1-4374-80b5-dce5d5575809</last><count>590</count></set></pubsub></iq>
Segmentation fault
[highfellow@web225 buddycloud-http-api]$
highfellow commented 11 years ago

I'm using node version 0.8.5 on a CentOS server. The libxml2 version is 2.9.0. The libxmljs version is 0.5.4. The http api server was cloned from the repository about 2 weeks ago. I've run the unit tests for libxml2, and it passes:

[highfellow@web225 libxmljs]$ npm test

> libxmljs@0.5.4 test /home/highfellow/build/buddycloud-http-api/node_modules/libxmljs
> node --expose_gc ./node_modules/nodeunit/bin/nodeunit test

Warning: program compiled against libxml 209 using older 206

attribute
✔ new

I'm thinking at the moment that if the error is a segfault, this is most likely to be happening outside of the server's javascript code, in one of the native libraries it calls. As far as I know, this means either libxml2 or libicu, which are called by libxmljs and node-stringprep.

Schnouki commented 11 years ago

Can you run node inside gdb (gdb node server.js) and get a backtrace (bt) once node segfaults?

Could you really check which version of libxml is used? ldd node_modules/libxmljs/build/Release/libxmljs.node

What was the error you got when compiling node-stringprep?

highfellow commented 11 years ago 

[highfellow@web225 buddycloud-http-api]$ ldd node_modules/libxmljs/build/Release/libxmljs.node
node_modules/libxmljs/build/Release/libxmljs.node: /usr/lib/libxml2.so.2: no version information available (required by node_modules/libxmljs/build/Release/libxmljs.node)
node_modules/libxmljs/build/Release/libxmljs.node: /usr/lib/libxml2.so.2: no version information available (required by node_modules/libxmljs/build/Release/libxmljs.node)
node_modules/libxmljs/build/Release/libxmljs.node: /usr/lib/libxml2.so.2: no version information available (required by node_modules/libxmljs/build/Release/libxmljs.node)
node_modules/libxmljs/build/Release/libxmljs.node: /usr/lib/libxml2.so.2: no version information available (required by node_modules/libxmljs/build/Release/libxmljs.node)
    linux-gate.so.1 =>  (0x0038a000)
    libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00205000)
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00955000)
    libm.so.6 => /lib/libm.so.6 (0x00db4000)
    libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x0046a000)
    libc.so.6 => /lib/libc.so.6 (0x00580000)
    libdl.so.2 => /lib/libdl.so.2 (0x00e60000)
    libz.so.1 => /lib/libz.so.1 (0x00110000)
    /lib/ld-linux.so.2 (0x00563000)

I assume this means it's using the version of libxml2 installed at server level, not the one I built manually and installed in my home directory.

The node-stringprep error looks like:

[highfellow@web225 build]$ npm install node-stringprep
npm http GET https://registry.npmjs.org/node-stringprep
npm http 304 https://registry.npmjs.org/node-stringprep

> node-stringprep@0.1.5 install /home/highfellow/build/node_modules/node-stringprep
> node-gyp rebuild

make: Entering directory `/home/highfellow/build/node_modules/node-stringprep/build'
  CXX(target) Release/obj.target/node-stringprep/node-stringprep.o
../node-stringprep.cc:89: error: expected ‘,’ or ‘...’ before ‘profileType’
../node-stringprep.cc:89: error: ISO C++ forbids declaration of ‘UStringPrepProfileType’ with no type
../node-stringprep.cc:157: error: use of enum ‘UStringPrepProfileType’ without previous declaration
../node-stringprep.cc: In static member function ‘static v8::Handle<v8::Value> StringPrep::New(const v8::Arguments&)’:
../node-stringprep.cc:62: error: ‘UStringPrepProfileType’ was not declared in this scope
../node-stringprep.cc:62: error: expected `;' before ‘profileType’
../node-stringprep.cc:65: error: ‘profileType’ was not declared in this scope
../node-stringprep.cc:72: error: ‘profileType’ was not declared in this scope
../node-stringprep.cc: In constructor ‘StringPrep::StringPrep(int)’:
../node-stringprep.cc:92: error: ‘profileType’ was not declared in this scope
../node-stringprep.cc:92: error: ‘usprep_openByType’ was not declared in this scope
../node-stringprep.cc: In static member function ‘static int StringPrep::parseProfileType(v8::String::Utf8Value&)’:
../node-stringprep.cc:161: error: ‘USPREP_RFC3491_NAMEPREP’ was not declared in this scope
../node-stringprep.cc:163: error: ‘USPREP_RFC3530_NFS4_CS_PREP’ was not declared in this scope
../node-stringprep.cc:165: error: ‘USPREP_RFC3530_NFS4_CS_PREP_CI’ was not declared in this scope
../node-stringprep.cc:167: error: ‘USPREP_RFC3530_NFS4_CIS_PREP’ was not declared in this scope
../node-stringprep.cc:169: error: ‘USPREP_RFC3530_NFS4_MIXED_PREP_PREFIX’ was not declared in this scope
../node-stringprep.cc:171: error: ‘USPREP_RFC3530_NFS4_MIXED_PREP_SUFFIX’ was not declared in this scope
../node-stringprep.cc:173: error: ‘USPREP_RFC3722_ISCSI’ was not declared in this scope
../node-stringprep.cc:175: error: ‘USPREP_RFC3920_NODEPREP’ was not declared in this scope
../node-stringprep.cc:177: error: ‘USPREP_RFC3920_RESOURCEPREP’ was not declared in this scope
../node-stringprep.cc:179: error: ‘USPREP_RFC4011_MIB’ was not declared in this scope
../node-stringprep.cc:181: error: ‘USPREP_RFC4013_SASLPREP’ was not declared in this scope
../node-stringprep.cc:183: error: ‘USPREP_RFC4505_TRACE’ was not declared in this scope
../node-stringprep.cc:185: error: ‘USPREP_RFC4518_LDAP’ was not declared in this scope
../node-stringprep.cc:187: error: ‘USPREP_RFC4518_LDAP_CI’ was not declared in this scope
make: *** [Release/obj.target/node-stringprep/node-stringprep.o] Error 1
make: Leaving directory `/home/highfellow/build/node_modules/node-stringprep/build'
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/home/highfellow/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:215:23)
gyp ERR! stack     at ChildProcess.EventEmitter.emit (events.js:91:17)
gyp ERR! stack     at Process._handle.onexit (child_process.js:674:10)
gyp ERR! System Linux 2.6.18-274.18.1.el5PAE
gyp ERR! command "node" "/home/highfellow/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/highfellow/build/node_modules/node-stringprep
gyp ERR! node -v v0.8.5
gyp ERR! node-gyp -v v0.6.3
gyp ERR! not ok 
npm ERR! node-stringprep@0.1.5 install: `node-gyp rebuild`
npm ERR! `sh "-c" "node-gyp rebuild"` failed with 1
npm ERR! 
npm ERR! Failed at the node-stringprep@0.1.5 install script.
npm ERR! This is most likely a problem with the node-stringprep package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-gyp rebuild
npm ERR! You can get their info via:
npm ERR!     npm owner ls node-stringprep
npm ERR! There is likely additional logging output above.

npm ERR! System Linux 2.6.18-274.18.1.el5PAE
npm ERR! command "/home/highfellow/bin/node" "/home/highfellow/bin/npm" "install" "node-stringprep"
npm ERR! cwd /home/highfellow/build
npm ERR! node -v v0.8.5
npm ERR! npm -v 1.1.46
npm ERR! code ELIFECYCLE
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /home/highfellow

The main difference between versions 0.1.4 and 0.1.5 of node-stringprep is in this commit, where it starts using compiler flags from icu-config:

https://github.com/astro/node-stringprep/commit/28ba2705b9d108a36c995230957b7f4b0a5b4499

I think the answer to this might be to install a newer version of libicu, but I'll have to leave this, and the gdb stuff, till later. (Current version is 4.8.1.1)

highfellow commented 11 years ago

Thanks for the help by the way :-)

Schnouki commented 11 years ago

You're welcome ;)

If you installed libxml and friends in your $HOME, could you try setting LD_LIBRARY_PATH to an appropriate path:

export LD_LIBRARY_PATH=$HOME/path/to/lib:$LD_LIBRARY_PATH
ldd node_modules/libxmljs/build/Release/libxmljs.node
ldd node_modules/node-stringprep/build/Release/node-stringprep.node
node server.js

and maybe also ldd node_modules/node-stringprep/build/Release/node-stringprep.node without setting LD_LIBRARY_PATH...

highfellow commented 11 years ago 
[highfellow@web225 default]$ ldd node-stringprep.node 
    linux-gate.so.1 =>  (0x00507000)
    libpthread.so.0 => /lib/libpthread.so.0 (0x00745000)
    libdl.so.2 => /lib/libdl.so.2 (0x0084d000)
    libicui18n.so.48 => not found
    libicuuc.so.48 => not found
    libicudata.so.48 => not found
    libm.so.6 => /lib/libm.so.6 (0x007ad000)
    libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00cca000)
    libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00a00000)
    libc.so.6 => /lib/libc.so.6 (0x00110000)
    /lib/ld-linux.so.2 (0x00563000)

This might well explain the problem! Starting the api server with LD_LIBRARY_PATH set stops it crashing, and also stops the two error messages the server was giving on startup (pasted below):

Cannot load StringPrep-0.1.0 bindings. You may need to `npm install node-stringprep'
Warning: program compiled against libxml 209 using older 206

The weird thing is the xmpp server has been giving the StringPrep error for ages, but still seems to run OK.

All now seems to be working as it should :-).

Schnouki commented 11 years ago

Great! These messages were bothering me. I'm glad it works :-) (For the record, I think the issue was with libxml, with some ABI breakage between the version used at compile-time and the one used at runtime. node-xmpp apparently ignores errors caused by node-stringprep not being present)

highfellow commented 11 years ago

Just wanted to add that setting LD_LIBRARY_PATH doesn't cure the node-stringprep-0.1.5 build error. Setting the node-stringprep version to 0.1.4 in package.json is a temporary fix, which gets the server to build OK. I'm just about to reinstall libicu to see if that fixes this.

highfellow commented 11 years ago

I had trouble building the new libicu on the CentOS 5 server I'm using for my web hosting. I'll leave this for now.

imaginator commented 11 years ago

@highfellow did everything work out for you? Are you now able to run the webclient and API server?

highfellow commented 11 years ago

No, I still haven't got the web client running - see:

https://github.com/buddycloud/webclient/issues/66

I could still do with some help with this if anyone has any ideas why it's not working?

ChrisMohritz commented 11 years ago

if it helps...i just ran into the 0.1.5 node-stringprep error with a clean install of ubuntu 12.10 using the latest master branch of bc server (& libicu48). installing libicu-dev cleared up the node-stringprep error during bc install.

imaginator commented 11 years ago

Thanks Chris. Added to https://buddycloud.org/wiki/Install#Install_2