allow to raise priority of the master thread

[GoogleCodeExporter]

What steps will reproduce the problem?
1. set a "low" number of worker threads,
2. wake them by issuing requests,
3. once the number of threads is almost filled the processing of requests will 
slow down, as the master thread is not waking worker threads on time, as they 
are busy, but master and workers have the same priority

What is the expected output? What do you see instead?
expected output is the prompt waking of worker threads

What version of the product are you using? On what operating system?
Linux, XP

Please provide any additional information below.

I currently do this:
static void master_thread(struct mg_context *ctx) {
  fd_set read_set;
  struct timeval tv;
  struct socket *sp;
  int max_fd;

#if defined(_WIN32)
  SetThreadPriority(GetCurrentThread(), THREAD_PRIORITY_ABOVE_NORMAL);
#endif // _WIN32

which solves the problem under windows, but, for linuces/unices a pthreads call 
would be necessary. Perhaps the author can do something about this?

Original issue reported on code.google.com by janez...@gmail.com on 20 Feb 2012 at 9:23

[GoogleCodeExporter]

Believe it or not, I am currently trying to solve exactly this problem.
No solution for Unix so far.

Original comment by valenok on 20 Feb 2012 at 10:56

• Changed state: Accepted

[GoogleCodeExporter]

How about using:

int pthread_setschedprio(pthread_t thread, int prio);

also, check out:

int sched_get_priority_max(int policy);

int sched_get_priority_min(int policy);

Original comment by janez...@gmail.com on 20 Feb 2012 at 11:49

[GoogleCodeExporter]

You can also extend the pthreads implementation you have.

Original comment by janez...@gmail.com on 20 Feb 2012 at 11:50

[GoogleCodeExporter]

I'd try:

pthread_setschedprio(pthread_self(), -1);

Original comment by janez...@gmail.com on 20 Feb 2012 at 12:01

[GoogleCodeExporter]

Tried this:

  // Increase priority of the master thread
  struct sched_param sched_param;
  int policy;

  pthread_getschedparam(pthread_self(), &policy, &sched_param);
  sched_param.sched_priority = sched_get_priority_max(policy);
  pthread_setschedparam(pthread_self(), policy, &sched_param);

The result is almost neutral, slightly worse then a baseline.
Tested with:
ab -c 50 -n 10000 http://127.0.0.1:8080/mongoose.c

Original comment by valenok on 20 Feb 2012 at 6:36

[GoogleCodeExporter]

Yeah, I was testing on MacOS Snow Leopard.
Going to test on Linux. If Linux test shows no difference, I am not going to do 
anything.

Original comment by valenok on 20 Feb 2012 at 6:40

[GoogleCodeExporter]

Submitted to the HEAD, you can test it yourself:
https://code.google.com/p/mongoose/source/detail?r=fc4339bcd1

Original comment by valenok on 20 Feb 2012 at 6:45

[GoogleCodeExporter]

Your results are false, there should be no difference whatsoever. My cgdb 
session shows:

(gdb) p sched_param
$4 = {__sched_priority = 1942065232}
(gdb) 
(gdb) p sched_param
$5 = {__sched_priority = 0}
(gdb) 
[New Thread 0x7ffff77ce700 (LWP 3433)]
(gdb) p sched_param
$6 = {__sched_priority = 0}

After stepping thought the 3 lines of code you wrote. You don't change the 
priority at all (i.e. sched_get_priority_max(policy) returns 0).

Can you please test with:
pthread_setschedprio(pthread_self(), -1);

And see how it performs?

Original comment by janez...@gmail.com on 20 Feb 2012 at 10:51

[GoogleCodeExporter]

pthread_setschedprio() is not present at all on my system.
Here's my gdb session:

lsm-macbookpro:mongoose lsm$ cc -g mongoose.c main.c -o mongoose -DISSUE_317
lsm-macbookpro:mongoose lsm$ gdb mongoose
GNU gdb 6.3.50-20050815 (Apple version gdb-1515) (Sat Jan 15 08:33:48 UTC 2011)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin"...Reading symbols for shared 
libraries .. done

(gdb) b master_thread 
Breakpoint 1 at 0x10000baeb: file mongoose.c, line 4036.
(gdb) r
Starting program: /Users/lsm/mongoose/mongoose 
Reading symbols for shared libraries +. done
[Switching to process 66607]

Breakpoint 1, master_thread (ctx=0x100800000) at mongoose.c:4036
4036      pthread_getschedparam(pthread_self(), &policy, &sched_param);
(gdb) display sched_param 
1: sched_param = {
  sched_priority = 0, 
  __opaque = "\000\000\000"
}
(gdb) n
Mongoose web server v. 3.1 started on port(s) 8080 with web root [.]
4037      sched_param.sched_priority = sched_get_priority_max(policy);
1: sched_param = {
  sched_priority = 31, 
  __opaque = "\n\000\000"
}
(gdb) n
4038      pthread_setschedparam(pthread_self(), policy, &sched_param);
1: sched_param = {
  sched_priority = 47, 
  __opaque = "\n\000\000"
}

As you can see, priority does indeed change.
Your debugger session suggests me that on your system priority 0 is the 
maximum, or sched_get_priority_max() for some reason does not return the right 
value.

And please do not claim that results are false, they are what they are. I've 
published my code, environment and my testing method, you're welcome to repeat.

Original comment by valenok on 20 Feb 2012 at 11:05

[GoogleCodeExporter]

Try this in your benchmark:

 struct sched_param sched_param;
 sched_param.sched_priority = sched_get_priority_max(SCHED_RR);
 pthread_setschedparam(pthread_self(), SCHED_RR, &sched_param);

I am still not convinced that the maximum priority is the way to go. We only 
need one priority above the default.

Original comment by janez...@gmail.com on 20 Feb 2012 at 11:08

[GoogleCodeExporter]

Ok, I was unfair to assert the falseness of your results, but the same problem 
would have hit you on linux. It is because threads are created in the 
SCHED_OTHER policy class by default and this policy class has no priorities.

reference:
http://stackoverflow.com/questions/3140505/pthread-setschedprio-fails-with-einva
l

Original comment by janez...@gmail.com on 20 Feb 2012 at 11:11

[GoogleCodeExporter]

That is, I'd also test with:

 sched_param.sched_priority = sched_get_priority_min(SCHED_RR) + 1;

I can only test on Windows, as the issue is only present there (the software is 
win32 API only) and raising the priority there solved the issue. But I can 
debug on linux.

Original comment by janez...@gmail.com on 20 Feb 2012 at 11:18

[GoogleCodeExporter]

Yeah, that might be the cause.
I've tried multiple combinations -- SCHED_RR, SCHED_FIFO, with maximum priority 
or just one above the default. I see no difference in the benchmark.
I've repeated the tests with the larger file,
$ ab -c 100 -n 5000 http://127.0.0.1:8080/gmon.out
$ ls -l gmon.out
-rw-r--r--  1 lsm  5000  706549 Feb 14 17:59 gmon.out

I still see no difference.
Maybe on linux/BSD it has an effect, will see. Neutral on Macos 10.6

Original comment by valenok on 20 Feb 2012 at 11:22

[GoogleCodeExporter]

Oh. Interesting. I'll definitely include your windows code then.

Original comment by valenok on 20 Feb 2012 at 11:24

[GoogleCodeExporter]

Again, the issue appears only when almost all worker threads are exhausted 
(pumping long files, for example). The issue appeared when I implemented a 
small service framework on top of mongoose, for some ajax things. One ajax 
request starts a stream (potentially never ending one) of events. The thread 
count was initially 16, but as the number of available threads decreased (due 
to hijacked threads by the event service), the transfer speed decreased also, 
as the hijacked threads were pumping out events and only, say, 1 worker thread 
remained available to service the clients (say with files). Still, this 1 
worker thread did its job fast enough when the priority of the master thread 
was raised (by 1) so that it woke up immediately after a request was received 
and did not have to wait for the workers to finish pumping their events.

Maybe you can test my fix on WIN32 and leave it if the test proves favorable.

Original comment by janez...@gmail.com on 20 Feb 2012 at 11:31

[GoogleCodeExporter]

Oh, and all the action was done on the loopback interface, the events were fft 
results from an audio player, i.e. a lot of data. Mongoose stopped serving 
files effectively when only a few worker threads remained, but not through slow 
transfer speeds (loopback is fast), but simply because of a strange delay 
between receiving a request for a file and actually starting to serve it. Hence 
I figured it must be a problem with waking up worker threads and so it was.

Original comment by janez...@gmail.com on 20 Feb 2012 at 11:37

[GoogleCodeExporter]

I suggest this code be used for tests:

    struct sched_param sched_param = {
      sched_get_priority_min(SCHED_RR) + 1
      };

    pthread_setschedparam(pthread_self(), SCHED_RR, &sched_param);

with uid=0 (root), as privilege is required to set priority under linux.

Original comment by janez...@gmail.com on 9 Mar 2012 at 11:39

[GoogleCodeExporter]

The other alternative would be to do this:

    struct sched_param sched_param = {
      sched_get_priority_min(SCHED_BATCH)
      };

    pthread_setschedparam(pthread_self(), SCHED_BATCH, &sched_param);

In worker threads, as SCHED_BATCH does not require any privilege, but mildly 
disfavors the thread.

Original comment by janez...@gmail.com on 9 Mar 2012 at 11:55

[GoogleCodeExporter]

Please build with -DISSUE_317.

Original comment by valenok on 22 Sep 2012 at 2:53

• Changed state: Fixed