haraka / Haraka

A fast, highly extensible, and event driven SMTP server
https://haraka.github.io
MIT License
5.02k stars 662 forks source link

Running a blocking syncronous call will hang up this connection or all connections? #1286

Closed ricardopolo closed 8 years ago

ricardopolo commented 8 years ago

I created this question in stackoverflow thinking specifically in Haraka http://stackoverflow.com/questions/34503840/using-a-blocking-long-sync-operation-will-pause-all-users-in-node-js

In a code like this in a haraka plugin

exports.something = function (next, connection) {
  connection.transaction.add_body_filter('', function (content_type, encoding, body_buffer){
    var header = connection.transaction.header.get("header");
    var url = 'https://server.com/api/something?header='+header;
    var xhr = new XMLHttpRequest();
    xhr.open("GET", url, false);
    xhr.send();
   return new Buffer(xhr.responseText)
  });

  return next();
}

It will lock this current connection and slow down this email or all emails will be locked until the request is finished?

smfreegard commented 8 years ago

It will lock this current connection and slow down this email or all emails will be locked until the request is finished?

It will block the server completely until it completes - that means all current and pending connections to that child process. e.g. if you're running Haraka with nodes=cpus (if you aren't you should be), then it will block that child completely and all other connections that child is handling. The other child processes will be unaffected though.

You should NEVER use synchronous functions in node.js applications like Haraka. If you're tempted to do so - then you're doing it wrong. The only place for them is in scripts that are run interactively like a shell-script written in node.js.

I don't think you're going to be able to do what you appear to be attempting to do based on the way Haraka currently works.

The body filter must be configured before Haraka starts the data stream (e.g. it must happen at or before hook_data) as it has to buffer the whole message in memory to be able to modify it, so you can't switch in different behaviours based on message headers as they come in as part of the data which is therefore too late.

Modifying message bodies efficiently is pretty difficult and the existing code to do so is a big compromise.

ricardopolo commented 8 years ago

@smfreegard thanks for your answer. The code that I show works, it's blocking but works!!!

So I think there may be a way. I mean when then Body Filter runs, the whole message is in memory and we have access to headers.
Maybe if just changing the body filters to accept a callback instead of just return a Buffer... What do you think? Am I completely wrong?

Thanks again

ricardopolo commented 8 years ago

Maybe another way that I think is to have a new event called 'before_datapost' that run when you have all lines buffered but don't want to start the stream yet