Watson.Lite - Server auto-disposing itself after ~24 hours of intensive usage.

[GitHubProUser67]

Hello, with the last version of the nuget package, there is a very nasty issue that happens after running the server for a very long time.

I found out that the server puts itself in a "closed" state so when doing a reboot to check, the server thrown this exception:

It explicitly mention that the server is not listening.

It seems this problem started to happen recently after the most recent updates. But the problem is I see no exception being thrown by the library, despite showing hidden exceptions...

I would like very much your help on this, this is a critical problem.

Thanks in advance.

[GitHubProUser67]

I started using the classic Watson to see if the issue also happens with HTTPListener.

Note that I use a custom version based on an other implementation of HTTPListener which supports SSL, I will PR it when this custom implementation has it officially on the repo (https://github.com/space-wizards/SpaceWizards.HttpListener/pulls).

[jchristn]

Hi @GitHubProUser67 when you say a really long time, are you referring to hours, days, weeks, months? I have instances deployed in production of both Watson and Watson.Lite and haven't encountered this issue. Any steps you could provide to reproduce it would be helpful, or an indication of time and type/volume of traffic or any other condition that might be a cause of the problem.

Also when you say so when doing a reboot to check, the server thrown this exception, what steps are you taking here?

Cheers, Joel

[GitHubProUser67]

Hello jchristn, I want to thank you for taking time to help me on this,

When it happens, it usually happens after 26 hours or around 1 day and half.

What happens when the server stops, is that the server throws no exception at all, but simply stop listening by itself.

To reproduce the issue, simply download this version of the HTTPSecureServerLite server here https://github.com/GitHubProUser67/MultiServer3/commit/8f0df3c42a3561331876ea76b8d7a920e18a61ca

Let it sit for a good 24+ hours and do a request, you will see it will not accept anything after a little while.

And when trying to reboot the server via command line (any key to enable the action, then R to reboot, then Y to confirm), you will see it will throw the "WebServer is already stopped" exception.

Almost as if something shutdown the server internally. Currently testing the normal WebServer to see if it is related to CavemanTcp.

[jchristn]

Thanks, I'll test first on my side to see if I can reproduce in isolation. What kind of load are you passing to the server to cause this, or, does it happen even if the server sits idle?

[GitHubProUser67]

So, there is a lot of load of any types while the server is online, mainly file loading/servicing.

I didn't fully take time to see what it does when being fully idle, I always used powershell to simulate some load. Will check when possible.

[GitHubProUser67]

Yep, to be more precise serve a file of aprox 10mb and do a script that batch load it for a long time.

[GitHubProUser67]

So it is confirmed that the HTTPListener backend doesn't do ths problem.

[jchristn]

Hi @GitHubProUser67 I've been running a Watson.Lite instance running a variety of requests with variable delay and variable request body and response body sizes for the past 4 days and have yet to see any issue. I'm afraid I'll need a little more telemetry data on how to reproduce this in isolation. I unfortunately can't peel away enough time to do this within your code base.

[jchristn]

Closing as I can't reproduce this.