keithbloom / fluorinefx

Automatically exported from code.google.com/p/fluorinefx
0 stars 0 forks source link

High CPU load, RTMP service stops #9

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago
What steps will reproduce the problem?
There is no easy way to reproduce the problem. I have a RTMP-Application
with a few hundred connected clients. Those clients invoke my application
but the application also invokes the client. After some hours a thread of
fluorineFx suddenly runs at 100% load and i found out that it happens when
i do a "IConnection.invoke" (the server calls the client). Sometimes this
"lock" goes away after some time, and sometimes it stays forever. As soon
as the problem happens, i can't do any additionals "IConnection.invoke"
calls anymore. So i think there is a lock (spin-wait?) that blocks all calls.

What is the expected output? What do you see instead?
I single "IConnection.invoke" should not stop the whole Application from
working.

What version of the product are you using? On what operating system?
10.0.0.17 (SVN-latest version).
Windows Server 2008, 64Bit

Please provide any additional information below.
I attach 2 logfiles. The logs are quite big, the #[number] tells the text
line inside the logfile. 

You see a "Write RTMP Packet" message (this is the moment when the 100% CPU
load of the thread starts running) and a "Connection reset by peer, error
code was 10054" message (this is the moment when the 100% CPU load of the
thread stops). 

I found out that the thread with the 100% load is not the same thread as
the one that does the "Write RTMP Packet", i think it's a worker thread
that does all RTMP traffic managment that is blocked (with 100% load) as
soon as this happens.

==========================================================
Example 1 (~ 80 seconds high load)

#48997 2009-12-05 13:22:25,607 [31] DEBUG
FluorineFx.Messaging.Rtmp.RtmpServerConnection -
f91dfad323e03699840d07f8a37f94b Write RTMP Packet
#49038 2009-12-05 13:22:25,607 [31] DEBUG
FluorineFx.Messaging.Rtmp.RtmpServerConnection -
f91dfad323e03699840d07f8a37f94b Socket send
#205284 2009-12-05 13:23:45,604 [31] DEBUG
FluorineFx.Messaging.Rtmp.RtmpServerConnection -
f91dfad323e03699840d07f8a37f94b Connection reset by peer, error code was 10054

==========================================================
Example 2 (~ 120 seconds high load)

#3538  2009-12-05 13:36:22,864 [22] DEBUG
FluorineFx.Messaging.Rtmp.RtmpServerConnection -
c3cac68fa815616b3ef3d193427fef7 Write RTMP Packet
#3579  2009-12-05 13:36:22,864 [22] DEBUG
FluorineFx.Messaging.Rtmp.RtmpServerConnection -
c3cac68fa815616b3ef3d193427fef7 Socket send
#231895 2009-12-05 13:38:23,359 [22] DEBUG
FluorineFx.Messaging.Rtmp.RtmpServerConnection -
c3cac68fa815616b3ef3d193427fef7 Connection reset by peer, error code was 10054

==========================================================

This problem stops me from using FluorineFX and i have to use WebOrb - and
i hate it. WebOrb has a lot of other issues.

Thank you very much,
Bernhard

Original issue reported on code.google.com by bernhard...@gmail.com on 16 Jan 2010 at 12:17

Attachments: