danbarua / NEventSocket

A reactive FreeSwitch eventsocket library for Modern .Net
Mozilla Public License 2.0
74 stars 37 forks source link

Dispose issue #28

Closed mguerrieri closed 8 years ago

mguerrieri commented 8 years ago

Using the DtmfTest() merthod as an example, if I add

await Task.Delay(1000);

right after the InboundSocket.Connect() call, then when the Task.Deplay call begins, the log shows

2016-02-08 04:16:22 TID[21] TRACE Buffer Read Observable Completed 0 chunks left.

and any additional calls fail because "received" is disposed and null.

Is this not a supported scenario? The application I am trying to use the library with involves opening an inbound socket and subscribing to events then just sleeping until events are received on the socket or an external trigger requires an origination call to be made. In this scenario, the socket opens and authenticates, then immediately dies (which I am simulating with the Task.Delay() call in the sample application.)

danbarua commented 8 years ago

Ok cool, so this is what happens when the socket is shut down - either we closed the socket or FreeSwitch did. Is there any chance you can reproduce the issue with a minimum console application and email me the code to mygithubusername@gmail.com?

mguerrieri commented 8 years ago

I can if you really need me to, but all I did was take your sample code, calling the DtmfTest() method, and added the delay (and changed the server host and extension to work with my setup). Also added the client.SubscribeEvents() as per your suggestion on my other issue, but this issue occurs regardless. If you comment out the Task.Delay(), it works as expected. If not, the socket dies. If you still want a sample app, let me know...

    private static async void DtmfTest()
    {
        var client = await InboundSocket.Connect("myswitch", 8021, "ClueCon");

        await Task.Delay(1000); // Added this

        await client.SubscribeEvents();

        var originate =
            await
            client.Originate(
                "user/1001",
                new OriginateOptions
                {
                    CallerIdNumber = "123456789", 
                    CallerIdName = "Dan Leg A",
                    HangupAfterBridge = false,
                    TimeoutSeconds = 20
                });
senpark15 commented 8 years ago

Hi, I am also having the same issue. I am using socket.SendApi("show calls count") method to get current calls count. First few times it gives the result. but once this occured TRACE Buffer Read Observable Completed 0 chunks left. , all subsequent calls fails. please suggest me any workaround

dchristoffersen-vivint commented 8 years ago

I am seeing this exact same behavior. Here is what I find odd. I can connect to my test freeswitch server and subscribe to events and it works great. My production instance of freeswitch will keep the inboundsocket open indefinitely. However, the instant I subscribe to events it closes the socket and throws the following error:

[ERROR] - Error waiting for Command Reply to [event plain CHANNEL_EXECUTE_COMPLE TE BACKGROUND_JOB CHANNEL_HANGUP CHANNEL_ANSWER CHANNEL_PROGRESS CHANNEL_PROGRES S_MEDIA CHANNEL_BRIDGE CHANNEL_UNBRIDGE CUSTOM conference::maintenance]. System.ObjectDisposedException: The collection has been disposed. Object name: 'BlockingCollection'. at System.Collections.Concurrent.BlockingCollection1.CheckDisposed() at System.Collections.Concurrent.BlockingCollection1.get_IsCompleted() at System.Collections.Concurrent.BlockingCollection`1.<GetConsumingEnumerable

d_2.MoveNext() at System.Reactive.Linq.ObservableImpl.ToObservable`1..Loop(IEnumerator`1 en umerator, ICancelable cancel) in c:\Code\ESL\NEventSocket\src\NEventSocket\FreeS witch\AttendedTransferResult.cs:line 0

[ERROR] - Failed to generate log message System.NullReferenceException: Object reference not set to an instance of an obj ect. at NEventSocket.Sockets.ObservableSocket.<.ctor>b2() in c:\Code\ESL\NEventS ocket\src\NEventSocket\Sockets\ObservableSocket.cs:line 66 at NEventSocket.Logging.LoggerExecutionWrapper.<>cDisplayClass2.b__0() in c:\Code\ESL\NEventSocket\src\NEventSocket\Logging\LibLog.cs:line 467

dchristoffersen-vivint commented 8 years ago

The problem is that the command is timing out. How do we increase the timeout for these commands?

danbarua commented 8 years ago

Hi everyone,

I'm working on removing the Rx magic from the lower layers and getting a bit closer to the metal so we can see what's going on - the current master branch is getting there but it buckles under heavy load (when I fire up 300 inboundsockets and try to subscribe to heartbeat events, I get weird multithreading issues.)

It would be great if anyone can try the code out and report back to me any issues.

danbarua commented 8 years ago

I have mostly been using NEventSocket for OutboundSocket projects at work, which is why I have missed the issues with InboundSocket - your feedback is invaluable on this so thank you for your patience.

dchristoffersen-vivint commented 8 years ago

That's great. I am more than happy to help any way I can. We have noticed that our app needs to cycle the inboundsocket connection every few hours in order to manage threads. Somehow reactive is not disposing of threads under high load. We only see this when we are processing a high volume of events 10,000 plus per hour.

danbarua commented 8 years ago

@mguerrieri @dchristoffersen-vivint @senpark15 Could you try running the latest master build and let me know if you have any issues? Thanks

dchristoffersen-vivint commented 8 years ago

I just grabbed it. I am building and deploying the code to test it. I will let you know what I find.

dchristoffersen-vivint commented 8 years ago

I have been running the latest to catch our production events. It seems to be handling everything well. I did notice that it threw a new exception that you coded for: IOException - Error reading from the stream. This error closed the socket connection. Is that what you expected it to do? I have not seen this happen before in other logging. I will continue testing.

dchristoffersen-vivint commented 8 years ago

We have it out in Production. So far it is looking good. We are going to run it over night and I will let you know how it looks in the morning.

danbarua commented 8 years ago

I did notice that it threw a new exception that you coded for: IOException - Error reading from the stream. This error closed the socket connection. Is that what you expected it to do?

Yes this is correct - the socket will log the error and shut down when we get IOException or SocketException.

danbarua commented 8 years ago

I have set up a NuGet feed on MyGet for pre-release builds - any commit on the master branch will now produce a new package which will be available here: NuGet v2(VS2012) NuGet v3 (VS2015)

dchristoffersen-vivint commented 8 years ago

We have been running the de73971 commit from master for the past 17 hours with no problems. The threads are being disposed of and everything looks clean. I think it is fixed.

danbarua commented 8 years ago

Great - re-open this issue if you have any further problems, thanks for your feedback!