Prodigy slows down processes with a lot of logging output

[shosti]

For a slow rails app with response times of ~17s when running the server from the terminal, responses went to ~50s when running the server through prodigy. I thought it might be logging-related, so I disabled logging entirely (with (setq prodigy-process-on-output-hook nil)), but the results were the same. I'm not sure whether this is necessarily solvable or an inherent problem with Emacs inferior processes, but it's probably worth looking into.

[rejeep]

That sure is a slow app! :)

I have tried out three different Rails apps with Prodigy and not noticed that it is any slower.

• What version of Emacs are you using? I run Emacs 24.3.1 on OSX.
• Is the Rails app you are talking about open source so that I can try it out? Otherwise, maybe you can try to debug this a bit more on that app?
• Have you noticed that any other Rails app is slower via Prodigy?

I'm not an expert on processing, but I guess Prodigy processes will be sub-processes of the Emacs process? I don't know if requests to processes go via Emacs or directly to that process. Will have to read up a bit more on that.

[shosti]

Yeah, I've been diagnosing performance problems :) (not open source unfortunately). I'll look into it when I have a chance--just wanted to open the issue to track progress/discuss (I'm also running 24.3.1 on OSX).

[rejeep]

Ok, great. Maybe there are others with similar issues?

[magnars]

Well, I had a process that was outputting a lot of log messages - and it was terribly slow. But fixing that solved my issue.

[rejeep]

With https://github.com/rejeep/prodigy.el/issues/37 and https://github.com/rejeep/prodigy.el/issues/23, services that logs a lot of output should be handled quite well.

[shosti]

The issue does appear to be logging-related after all--I disabled stdout logging from the app, and the times went back to normal. However, the fact that it was slow even when disabling logging at the prodigy level makes me think it's related to the way Emacs deals with output from inferior processes, so I'm not sure if #37 or #23 will necessarily help that much (although it's worth trying, and they could still be useful).

[rejeep]

If you remove the set-process-filter line in prodigy, what happens then? If you remove that, Emacs should not care about output from processes (not sure if that is completely true though).

[shosti]

Yeah, with set-process-filter disabled but logging enabled at the app level, time went from ~5s in the console to ~18s in prodigy (I guess Emacs still does something with the output by default even without a process filter). Maybe the best way to deal with disabling/redirecting output would be to wrap processes in a mini shell script?

[rejeep]

Hmm, ok.

Emacs usually have decent documentation, so I think we should start there. Then if we cannot figure it out from that, we could always ask on their mailing list. Maybe someone has an idea.

[shosti]

Having worked with this for a while, I think this is best dealt with on the level of the service, rather than prodigy (e.g. by logging directly to a file from the service or wrapping in a shell script that redirects output).

[rejeep]

I have experienced that services running via Prodigy can be a little slower, but I cant figure out why they would other than it's limitations in Emacs, hence nothing we can do, except what you say.

Who knows, if Emacs ever adds support for threads, this might work better.