search

wandenberg / nginx-push-stream-module

A pure stream http push technology for your Nginx setup. Comet made easy and really scalable.
Other
2.22k stars 295 forks source link

possible memory leak in IE7 & IE8 on XP using pushstream.js #7

Closed texdev closed 12 years ago

texdev commented 12 years ago

I had a problem with another push module that caused nginx worker threads to crash. Using nginx-push-stream-module seems to have solved that problem (thanks!). However, when testing to confirm the fix, I ran into a memory leak using IE7 & IE8 (probably IE9, but not tested). I had four client machines, each ran multiple client browsers 1 IE, 1 Firefox, 1 Chrome for windows machines. I also ran firefox and chrome on a linux machine. All, except IE, survived an overnight test. The test was roughly on the order of 1 message every 2 seconds to each client browser, but the application is highly "bursty" with events coming all at once and then slacking off for a minute. In the morning all IE instances had failed. As mentioned above, firefox and chrome did just fine. I am using the provided javascript.js to manage long polling connections. (I then tried using "Forever iFrame" streams and observed the same results, although memory consumption seemed slower.) I came up with the following test to exhibit the problem:

  1. Load IE on your client machine and browse to chat_longpolling.html on your server.
  2. Open "windows task manager" on your client machine
  3. Select the "processes" tab and and order by "Mem Usage" descending.
  4. On the chat page make sure the 'example' room is selected.
  5. Run the bash script code provided below to generate messages. (modify hostname)
  6. Observe IE memory usage using "windows task manager"
  7. Note that memory usage varies up and down, but trends up.
  8. Note that I have only tested using windows XP
  9. Note that there should be a slight increase in memory usage because the message text increases the size of the display div, but the amount of memory consumed in IE far exceeds the size of the sum of the message text. My actual application overwrites the data, but I wanted to use your provided chat_longpolling.html code as the simplest way to show the problem.

Bash push script:

#!/bin/bash
c=1
while [ $c -le 5000000 ]
do
    echo "loop $c"
    curl http://<hostname>/pub?id=example -d "{\"nick\":\"loop\", \"text\":\"$c\"}"
    (( c++ ))
    sleep 1
done

I may have to write a custom javascript connection manager, but prefer to wait and use pushstream.js to stay compatible with future versions of nginx-push-stream-module. Please let me know if you observe the same memory leak issue.

Thanks, Lance

wandenberg commented 12 years ago

Hi Lance,

thanks for the feedback.

I will try to reproduce your tests and fix the problem.

I will keep you informed.

On Wed, Nov 30, 2011 at 8:55 PM, texdev < reply@reply.github.com

wrote:

I had a problem with another push module that caused nginx worker threads to crash. Using nginx-push-stream-module seems to have solved that problem (thanks!). However, when testing to confirm the fix, I ran into a memory leak using IE7 & IE8 (probably IE9, but not tested). I had four client machines, each ran multiple client browsers 1 IE, 1 Firefox, 1 Chrome for windows machines. I also ran firefox and chrome on a linux machine. All, except IE, survived an overnight test. The test was roughly on the order of 1 message every 2 seconds to each client browser, but the application is highly "bursty" with events coming all at once and then slacking off for a minute. In the morning all IE instances had failed. As mentioned above, firefox and chrome did just fine. I am using the provided javascript.js to manage long polling connections. (I then tried using "Forever iFrame" streams and observed the same results, although memory consumption seemed slower.) I came up with the followi ng test to exhibit the problem:

  1. Load IE on your client machine and browse to chat_longpolling.html on your server.
  2. Open "windows task manager" on your client machine
  3. Select the "processes" tab and and order by "Mem Usage" descending.
  4. On the chat page make sure the 'example' room is selected.
  5. Run the bash script code provided below to generate messages. (modify hostname)
  6. Observe IE memory usage using "windows task manager"
  7. Note that memory usage varies up and down, but trends up.
  8. Note that I have only tested using windows XP

Bash push script:

!/bin/bash

c=1 while [ $c -le 5000000 ] do echo "loop $c" curl http:///pub?id=example -d "{\"nick\":\"loop\", \"text\":\"$c\"}" (( c++ )) sleep 1 done

I may have to write a custom javascript connection manager, but prefer to wait and use pushstream.js to stay compatible with future versions of nginx-push-stream-module. Please let me know if you observe the same memory leak issue.

Thanks, Lance

Reply to this email directly or view it on GitHub: https://github.com/wandenberg/nginx-push-stream-module/issues/7

wandenberg commented 12 years ago

Hi,

take a look on last updates and check if is ok now.

I did some tests on IE (6, 7, 8) and after 8 hours publishing messages at 1 msg/sec the process have this behavior:

Start memory / End memory 10840K / 11664K 11080K / 13304K 13980K / 15208K

Changes where specially on the test page and some small changes on pushstream.js

Please, repeat your tests with long polling and streaming and let me know if you have any other problem.

Regards, Wandenberg

On Wed, Nov 30, 2011 at 10:00 PM, Wandenberg Peixoto wandenberg@gmail.comwrote:

Hi Lance,

thanks for the feedback.

I will try to reproduce your tests and fix the problem.

I will keep you informed.

On Wed, Nov 30, 2011 at 8:55 PM, texdev < reply@reply.github.com

wrote:

I had a problem with another push module that caused nginx worker threads to crash. Using nginx-push-stream-module seems to have solved that problem (thanks!). However, when testing to confirm the fix, I ran into a memory leak using IE7 & IE8 (probably IE9, but not tested). I had four client machines, each ran multiple client browsers 1 IE, 1 Firefox, 1 Chrome for windows machines. I also ran firefox and chrome on a linux machine. All, except IE, survived an overnight test. The test was roughly on the order of 1 message every 2 seconds to each client browser, but the application is highly "bursty" with events coming all at once and then slacking off for a minute. In the morning all IE instances had failed. As mentioned above, firefox and chrome did just fine. I am using the provided javascript.js to manage long polling connections. (I then tried using "Forever iFrame" streams and observed the same results, although memory consumption seemed slower.) I came up with the followi ng test to exhibit the problem:

  1. Load IE on your client machine and browse to chat_longpolling.html on your server.
  2. Open "windows task manager" on your client machine
  3. Select the "processes" tab and and order by "Mem Usage" descending.
  4. On the chat page make sure the 'example' room is selected.
  5. Run the bash script code provided below to generate messages. (modify hostname)
  6. Observe IE memory usage using "windows task manager"
  7. Note that memory usage varies up and down, but trends up.
  8. Note that I have only tested using windows XP

Bash push script:

!/bin/bash

c=1 while [ $c -le 5000000 ] do echo "loop $c" curl http:///pub?id=example -d "{\"nick\":\"loop\", \"text\":\"$c\"}" (( c++ )) sleep 1 done

I may have to write a custom javascript connection manager, but prefer to wait and use pushstream.js to stay compatible with future versions of nginx-push-stream-module. Please let me know if you observe the same memory leak issue.

Thanks, Lance

Reply to this email directly or view it on GitHub: https://github.com/wandenberg/nginx-push-stream-module/issues/7

texdev commented 12 years ago

I ran tests for twenty one hours using the "longpolling" connection mode. The results show that IE7,IE8, & IE9 increase memory usage over time, but stabilize after running for a while. After stabilization, memory usage fluctuates within a range of about 300K for our application, as memory is allocated and released. The total memory usage was different between browser versions and even instances of the same version. There is some (intentional) variation in the versions of flash and java plugins used on our test systems. That may account for some of the difference in total memory usage between browser types, since our application delivers the received messages to flash via the ExternalInterface api.

The changes to pushstream.js have definitely addressed the memory leak for the “longpolling” mode. The current tests ran using the same release of our code as used when I first reported the issue only changing the pushstream.js file. That previous failing test had one IE instance using over 2GB memory and crashed another system.

I plan to reconfigure and run tests using the “stream” mode sometime in the next few days and will post those results and, hopefully, close this issue.

Thanks for your prompt help in addressing this issue!

Lance