search

olupotd / mongoose

Automatically exported from code.google.com/p/mongoose
MIT License
0 stars 0 forks source link

I have been added the SCGI support for version 2.8 here #137

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
hi there,
I have added the support for SCGI protocol (see: http://www.python.ca/scgi/
for more info) to mongoose version 2.8. 

Here are some comments:
1. You can disable it on compiling time by define NO_SCGI flag.

2. Two new arguments are added: 'scgi_ext' just like cgi_ext and
'scgi_host' specify the scgi application host by a ip:port format or just
port if the scgi application is running on local machine. If 'scgi_host' is
not specified, the default address is: 127.0.0.1:4000.

3. Just like isapi_scgi for IIS and the cgi2scgi tool which is provided by
http://www.python.ca/scgi/releases/scgi-1.14.tar.gz officially, The Status
line translation is not implemented. e.g.: The scgi application MUST return
a "HTTP/1.1 200 OK" rather than "Status: 200".

4. For your convenience all changed codes are enclosed by a comment like:
// [[ by BaiYang - add scgi support @ YYYY-MM-DD
   changed codes ... 
// ]] by BaiYang
So you can sreach "by BaiYang - add scgi support" to browse all changes.

Original issue reported on code.google.com by baiyang@gmail.com on 23 Apr 2010 at 2:16

Attachments:

GoogleCodeExporter commented 9 years ago 
Here are some performance test result done by my Tinkpad T61 notebook (with 
core 2
due mobile 2.0GHz, the scgi application simply generate a "Hello World" page):

#ab -c 100 -n 5000 127.0.0.1/test.fastcall
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests

Server Software:
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /test.fastcall
Document Length:        38 bytes

Concurrency Level:      100
Time taken for tests:   2.672 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      585000 bytes
HTML transferred:       190000 bytes
Requests per second:    1871.35 [#/sec] (mean)
Time per request:       53.438 [ms] (mean)
Time per request:       0.534 [ms] (mean, across all concurrent requests)
Transfer rate:          213.82 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.7      0      16
Processing:    16   52  12.9     47      94
Waiting:        0   49  14.5     47      94
Total:         16   53  12.9     47      94

Percentage of the requests served within a certain time (ms)
  50%     47
  66%     63
  75%     63
  80%     63
  90%     63
  95%     78
  98%     78
  99%     78
 100%     94 (longest request)

And here is the config file for doing this test:
acl -0.0.0.0/0,+127.0.0.1
admin_uri /admin
scgi_ext fastcall
scgi_host 127.0.0.1:7924
dir_list no
error_log error.log
max_threads 200
ports 127.0.0.1:80
root www

Original comment by baiyang@gmail.com on 23 Apr 2010 at 2:39

GoogleCodeExporter commented 9 years ago 
Would you send a patch instead, please?
Thanks!

Original comment by valenok on 23 Apr 2010 at 2:59

GoogleCodeExporter commented 9 years ago 
Test results on their own do not tell much, could you include the results for 
static file serving and CGI serving, 
please?

Original comment by valenok on 23 Apr 2010 at 3:13

GoogleCodeExporter commented 9 years ago 
Hi valenok, 
Did you mean a "diff patch" like the attachment?

Original comment by baiyang@gmail.com on 23 Apr 2010 at 3:21

Attachments:

GoogleCodeExporter commented 9 years ago 
The performance test with a static "hello world" file:

#ab -c 100 -n 5000 127.0.0.1/test.html
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests

Server Software:
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /test.html
Document Length:        40 bytes

Concurrency Level:      100
Time taken for tests:   1.438 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      1335000 bytes
HTML transferred:       200000 bytes
Requests per second:    3478.26 [#/sec] (mean)
Time per request:       28.750 [ms] (mean)
Time per request:       0.287 [ms] (mean, across all concurrent requests)
Transfer rate:          906.93 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   1.3      0      16
Processing:    16   28   7.7     31      63
Waiting:        0   27   7.9     31      63
Total:         16   28   7.7     31      63

Percentage of the requests served within a certain time (ms)
  50%     31
  66%     31
  75%     31
  80%     31
  90%     31
  95%     31
  98%     47
  99%     63
 100%     63 (longest request)

Original comment by baiyang@gmail.com on 23 Apr 2010 at 3:24

GoogleCodeExporter commented 9 years ago 
Same scgi "Hello world" appliction through cgi to scgi bridge:

#ab -c 100 -n 5000 127.0.0.1/test.cgi
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests

Server Software:
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /test.cgi
Document Length:        38 bytes

Concurrency Level:      100
Time taken for tests:   43.766 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      675000 bytes
HTML transferred:       190000 bytes
Requests per second:    114.24 [#/sec] (mean)
Time per request:       875.313 [ms] (mean)
Time per request:       8.753 [ms] (mean, across all concurrent requests)
Transfer rate:          15.06 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   3.6      0      63
Processing:    31  871 311.4    906    1609
Waiting:       31  861 307.1    891    1594
Total:         31  872 311.2    906    1609

Percentage of the requests served within a certain time (ms)
  50%    906
  66%   1031
  75%   1094
  80%   1156
  90%   1250
  95%   1328
  98%   1438
  99%   1453
 100%   1609 (longest request)

======== The configuration here: ========
acl -0.0.0.0/0,+127.0.0.1
admin_uri /admin
cgi_interp cgi2scgi.exe
dir_list no
error_log error.log
max_threads 200
ports 127.0.0.1:80
root www

Original comment by baiyang@gmail.com on 23 Apr 2010 at 3:31

GoogleCodeExporter commented 9 years ago 
Performence test with a "Hello World" cgi program written in C:

#ab -c 100 -n 5000 127.0.0.1/test.cgi
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests

Server Software:
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /test.cgi
Document Length:        38 bytes

Concurrency Level:      100
Time taken for tests:   19.969 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      650000 bytes
HTML transferred:       190000 bytes
Requests per second:    250.39 [#/sec] (mean)
Time per request:       399.375 [ms] (mean)
Time per request:       3.994 [ms] (mean, across all concurrent requests)
Transfer rate:          31.79 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   4.3      0      78
Processing:    31  396 176.9    375    1219
Waiting:       31  387 175.9    375    1219
Total:         31  396 177.0    375    1219

Percentage of the requests served within a certain time (ms)
  50%    375
  66%    438
  75%    484
  80%    516
  90%    594
  95%    703
  98%    859
  99%   1047
 100%   1219 (longest request)

====== the config is: ======
acl -0.0.0.0/0,+127.0.0.1
admin_uri /admin
cgi_interp cgitest.exe
dir_list no
error_log error.log
max_threads 200
ports 127.0.0.1:80
root www

The cgitest is a very simple program like this:

const char * pRESPONSE = 
    "Status: 200\r\n"
    "Content-Type: text/html; charset=utf-8\r\n"
    "Content-Length: 38\r\n"
    "\r\n"
    "<html><body>hello world!</body></html>";

// use raw Write api to disable the CR LF translation
DWORD nDummy;
WriteFile((HANDLE)STD_OUTPUT_HANDLE, pRESPONSE, strlen(pRESPONSE), &nDummy, 
NULL);

return 0;

Original comment by baiyang@gmail.com on 23 Apr 2010 at 4:07

GoogleCodeExporter commented 9 years ago 
Thanks so much, this is impressive.

I plan to integrate your patch when I'm ready with current refactoring which 
slightly 
changes the API.

I think your functions to connect/send data could be generalized, e.g. 
mg_connect(), 
and be used for creating proxy-like apps.

Original comment by valenok on 2 May 2010 at 11:24

GoogleCodeExporter commented 9 years ago 
OK, it should be a nice thing if we can reuse our code more and more. :-)

PS: I have found a small "mistake" recently: According to other scgi/fastcgi
implementations (e.g.: apache, nginx, lighttpd, etc.), the url pattern (which is
specified by our 'scgi_ext' argument) is only a placeholder to trigger the
scgi/fastcgi actions. So we don't need to do the existing checking of the url 
before
connect to scgi backend.

Maybe we could raise this code block in the function 'analyze_request':

#if !defined(NO_SCGI)
    } else if (match_extension(path,
        conn->ctx->options[OPT_SCGI_EXTENSIONS])) {
        send_scgi(conn, path);
#endif /* NO_SCGI */

to somewhere before the file existing checking:

    } else if (mg_stat(path, &st) != 0) {
        send_error(conn, 404, "Not Found", "%s", "File not found");

?

Original comment by baiyang@gmail.com on 3 May 2010 at 6:07

GoogleCodeExporter commented 9 years ago 
Yes, sounds reasonable.
Can I ask you to make a patch again please, this time against the trunk?
And please follow the style guidelines, see http://www.freebsd.org/cgi/man.cgi?
query=style&sektion=9

If you do this, I'll do more close code review and when it'll be ready to go 
in, you'll get 
SVN access. Since that point on, you'll be able to commit on your own.

Original comment by valenok on 3 May 2010 at 1:03

GoogleCodeExporter commented 9 years ago 
I've moved Mongoose to Mercurial, to make life easier to everybody who wants to 
make 
changes.
I suggest go to http://code.google.com/p/mongoose/source/clones?repo=default , 
make a clone, and commit to the clone directly. I will be able to see and 
comment on 
the code. Once it is ready, it will be pushed to the central repo. No 
patches/emails 
anymore.

Original comment by valenok on 3 May 2010 at 11:38

GoogleCodeExporter commented 9 years ago 
Sure, I will make the new pache later this week.

Original comment by baiyang@gmail.com on 4 May 2010 at 8:50

GoogleCodeExporter commented 9 years ago 
ok, it's down to baiyang-scgi

Original comment by baiyang@gmail.com on 8 May 2010 at 11:05

GoogleCodeExporter commented 9 years ago 
In the clone settings, go to "Administer" -> "Source" -> "Allow non-members to 
review 
code", and save.
Also, please update your repo to the latest change.

Original comment by valenok on 8 May 2010 at 11:49

GoogleCodeExporter commented 9 years ago 
Ok, done. :-)

Original comment by baiyang@gmail.com on 8 May 2010 at 12:08