nodejuice crashing upon refresh :: "This type of response MUST NOT have a body."

[michaelsbradleyjr]

I've got an app that makes use of nodejuice (via sidekick) and it runs just fine on a VM I have hosted on a Xen box on my local network.

I have the exact same setup (linux distro, xen based vm, etc.) on a VPS hosted by Linode, and when a client attempts a manual page refresh, or when a refresh is prompted from the server (e.g. as a result of a file-save), nodejuice bombs with this error:

  Error: This type of response MUST NOT have a body.
  at ServerResponse.write (http:346:11)
  at /my/path/nodejuice/servers/sidekick.js:45:29
  at /my/path/nodejuice/library/utility.js:134:24
  at /my/path/nodejuice/library/utility.js:225:20
  at fs:52:23
  at node.js:176:9

I've tried using node.js v0.1.91 and some later versions. I've also tried a 64bit linux vs a 32bit, with no difference. And I've tried with a VPS hosted by a different company, with the same result too.

I'm thinking that it may be related to the timing of some buffer/callback such that it's not an issue on my local network (never seen the error locally) but which becomes a problem when the network latency is significantly higher.

Any thoughts?

[michaelsbradleyjr]

BTW, nodejuice is great!! I just want to be able to use it effectively on linux boxen running outside my local network. :)

[stephenlb]

Glad you like nodejuice! Sidekick responses should always have a response body (other than HEAD requests). Sidekick is a proxy with extra magic. I've tested NodeJuice on GoDaddy Virtual Servers, RackSpace Virtual Servers and Amazon Virtual Servers successfully. Node is less stable past version 0.1.91 though perhaps has gotten better at 0.1.97 version. I'll update NodeJuice for 0.1.97 version. Check back later today.

[stephenlb]

I've only tested node with EPoll and Select, and not kqueue, poll or libevent. Which are you using?

[michaelsbradleyjr]

Hmm, I guess I have to reveal now that I'm somewhat of a beginner .. are those options when building node from source? I'm using creationix's nvm to manage/use various versions of node.js, and I'm not using any special parameters for the builds. My VPS is ubuntu 10.04 32bit, with build-essential. Nothing exotic -- everything I've got installed is plain vanilla.

[stephenlb]

You are running on EPoll in that case. I'm pushing to github in a few moments some updates. Though seems like something else may be the cause of this wonky.

[stephenlb]

Pushed updates. Nodejuice is running here on a 10.04 Ubunutu 64bit version with lots of binary and text data flying around successfully.

[stephenlb]

The updates that were pushed run on node 0.1.97 version.

[michaelsbradleyjr]

I'm fairly certain now it's the links inside the page's head, e.g. for script and link tags (js and css files), which are causing the problem.

If I comment those out, I can hit nodejuice on my VPSs pretty hard, no problem, with both push and client-side refreshes. And I'm able to go directly at the node.js process running behind sidekick without any problems, even when that stuff is not commented out.

So it must be the way that the node.js process behind sidekick is interacting with sidekick, with respect to the links inside the head-tags being loaded sequentially. That behind-sidekick node.js process involves the express framework from visionmedia, as was mentioned in IRC earlier today.

Now, if I kill and restart the nodejuice-sidekick process, it can handle one page load without crashing (most of the time). When it crashes, it's mostly following a refresh (sometimes upon the first request).

I've tried looking at the server headers with wget and curl, but haven't found any smoking guns yet.

I will note, for thoroughness sake, that the page being requested is a haml template that express renders then sends to the client (thru nodejuice), and I think it serves up the rendered template as a stream, rather than rendering the whole thing and sending it as one big chunk. I could be wrong about that, though.

One last note: I can leave the script and link tags intact inside the head tags, and when I load the page with curl and wget, nodejuice doesn't crash, even if I load the url over and over again very rapidly. But those commands don't try to follow the links inside the head-tags recursively, by default.

Note again that no crash is produced in any case when my app is running on a server in my local network. So whatever's going wrong, it must be aggravated by latency and how long it's taking for a buffer to drain, or something along those lines.

[stephenlb]

What is the 'Content-Type' Header in the request for JS/CSS? They should match the content type mime like 'text/javascript' or 'application/x-javascript' or 'text/css' for example.

[michaelsbradleyjr]

Okay, getting closer ... After digging around and watching content-type related stuff more carefully, I noted this:

Whenever nodejuice crashes, the "ctype" reported in the console output of nodejuice is wrong. For example, it will report 'ctype: "text"' rather than 'ctype: "text/css"'.

Whenever it answers a request without crashing, the ctype is reported correctly.

[michaelsbradleyjr]

By the way, thank you for looking at this issue so promptly. It's appreciated, but I know you don't have a gazillion free hours to help me figure this out. :)

[michaelsbradleyjr]

alright, further narrowing it down, the crashes seem to coincide with 304 http status codes (which are related to the content-type not being reported in the same way, I think)

[stephenlb]

I'm diving back into the code. Possible that ctype of 'tex't is causing wonky. Sidekick only watches for and modifies "*/html" ctypes.

[michaelsbradleyjr]

Ah interesting; on the other hand, it seems so odd that it's only crashing with respect to my VPSs in the cloud.

In the exact same app context (same everything else), I'm unable to cause a 304 to be generated even with rapid repeat refreshes on the client and server sides.

[michaelsbradleyjr]

Okay for clarity's sake (sorry for so many comments):

On my local network ubuntu VM, I wiped out the kiwi package manager for node.js, and reinstalled it, then reinstalled express and some other packages using kiwi. Now the behavior on my local VM matches what I'm seeing out on my VPSs, i.e. nodejuice is crashing in both contexts, coincident with the 304 status codes.

It's not a solution to this problem, but I didn't want to lead you to chase any ghosts, if I can help it.

[stephenlb]

That's great to have reproducible effects on your local box. I'm trying something and will have a patch in a few moments based on this new information.

[stephenlb]

I've updated github with the patch. Try updating your checkout. Here is some cool reference info: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5 (based on caching)

[michaelsbradleyjr]

Hey, congrats on squashing the bug!! squash it's working now .. mostly ..

Firefox and Chrome work as they should.

With Safari 4 on my Macbook, there still seems to be an issue, but that may lie with a failure on Apple's part to implement the http spec exactly to spec, or something. I'll describe the behavior, which only happens with Safari 4 (in my experiments so far):

First load works, no problem; client-side refresh "hangs" the browser's rendering process, and the console output on the nodejuice side looks funny, but it doesn't crash; at that point only a manual second client-side refresh loads the page successfully.

Also, push-refreshes from the server "hang" the browser's rendering process, and at that point only a manual client-side refresh will load the page successfully.

[stephenlb]

Great to hear! The Safari bug sounds to be a new issue. I'll take a few moments to look at it. Heading home soon.

[stephenlb]

My VBox is out of order. I'll start again tomorrow. Thank you for finding the 304 bug! It may have helped a lot of other people.

[michaelsbradleyjr]

No .. thank you, Stephen. I appreciate your efforts, today and generally with nodejuice. It's really great stuff and I plan on using it for a long time coming.

Have a great day/evening .. see you around in IRC perhaps. :-D