dotnet core 1.1 web application all requests in pending state

[aleha84]

Year ago started to develop web service (WEB API) based on dotnet core 1.0 with push notifications ability. Used for this SignalR, because I was already familiar with this library from .net framework.

Unfortunately there wasn't any official release for this library under dncore, so i used this: "Microsoft.AspNetCore.SignalR.Server": "0.2.0-*" from https://dotnet.myget.org/F/aspnetcore-ci-dev/api/v3/index.json

All works fine as expected. After a few month of development migrated to dot net core 1.1. All was fine. The web application is used like a service (widget). Requests are sent to it from another web site (making initial requests, get js scripts, establishing websocket connection). A month ago first release was made to production server (windows server 2016). All worked fine as expected. Over 500-600 simultaneous connections were being handled.

After that time, a lot of work was made (mostly busines logic and replacing EF with Dapper). And it was discovered that locally on the dev environment no more than 10 websocket connections could be established. All other requests (GET requests) to the service are in pending state. If one connection is closed, then the next receives response.

If I remove establishing of websocket connection from js ($.connection.hub.start()), then all is fine, no pending requests. No exception, no errors or warnings in logs. IIS logs don't have any entries either, like no requests were made.

On production, when the release has been applied, the situation is similar. Just pending state on all requests.

If switching back to the branch without the new features (without Dapper), locally it still handles only 10 connections, but on production all is fine, all same 500-600 connections are stable.

Dev environment dotnet core sdk versions installed: 1.0.1, 1.0.4, 1.1.0, 1.1.5, 1.1.6, 2.0.2, 2.0.3, 2.1.2 Server dotnet core runtime versions installed: 1.0.4, 1.0.5, 1.0.8, 1.1.1, 1.1.2, 1.1.5, 2.0.3

Could it be possible that application created on 1.1.5 framework version will work incorrect with aspnetcore iis module version 2.0.3 installed?

deps.json main diff between working and failing version:

"Dapper/1.50.2": {
            "dependencies": {
              "System.Collections": "4.3.0",
              "System.Collections.Concurrent": "4.3.0",
              "System.Collections.NonGeneric": "4.3.0",
              "System.Data.SqlClient": "4.3.1",
              "System.Dynamic.Runtime": "4.3.0",
              "System.Linq": "4.3.0",
              "System.Reflection": "4.3.0",
              "System.Reflection.Emit": "4.3.0",
              "System.Reflection.Emit.Lightweight": "4.3.0",
              "System.Reflection.Extensions": "4.3.0",
              "System.Reflection.TypeExtensions": "4.3.0",
              "System.Runtime": "4.3.0",
              "System.Runtime.Extensions": "4.3.0",
              "System.Runtime.InteropServices": "4.3.0",
              "System.Text.RegularExpressions": "4.3.0",
              "System.Threading": "4.3.0",
              "System.Xml.XDocument": "4.3.0",
              "System.Xml.XmlDocument": "4.0.1"
            },
            "runtime": {
              "lib/netstandard1.3/Dapper.dll": {}
            },
            "compile": {
              "lib/netstandard1.3/Dapper.dll": {}
            }               
}
.....
"compile": {
          "ref/netstandard1.0/System.Reflection.Emit.Lightweight.dll": {}
        },
.....
"Dapper/1.50.2": {
      "type": "package",
      "serviceable": true,
      "sha512": "sha512-6VcCiiOxVhgMwgMCuQuaff8I5IyDamrzeOjuUH1iMExh8R5TJqwkVCpwdVJGQdwSCqWsQA/PerrrK7T6mONthQ=="
    },  
}

I created a separate branch to migrate from 1.1 to 2.0 and use Microsoft.AspNetCore.SignalR -Version 1.0.0-alpha2-final, but there is no direct compatibility, and lack of functionality (at least it's not possible to send a message to a specified range of connections ids). So lots of additional work and tests needed, and I'm not sure if it will help at all.

Here is a sample for reproduction. MainSite is .Net Framework project. Service is .Net Core 1.1 project Both must be hosted in IIS. MainSite binding: localhost:8080, Service binding: localhost:8081. On my current development machine, I can establish 3 websocket connectionsbefore I start receiving a failure. Installed DotNetCore.1.0.7_1.1.4-WindowsHosting.exe to enable ANCM in IIS. (DotNetCore.1.0.8_1.1.5-WindowsHosting can not be installed because of the bug) Steps to reproduce:

1. Open MainSite page. Open dev console. Current connected clients count should be logged.
2. Open MainSite in new tab once more. Current connected clients count should increment.
3. Open MainSite once more in new tab. Nothing happens.
4. Open dev console of current new opened tab, go to Netwark tab, Refresh page. See Pending Request. Open one more tab with MainSIte. In Netrowk tab connection to Hub or to widget should be in pending state.
5. Close second tab, third tab should recieve response.

[analogrelay]

The 0.2.0 version you referenced above has never been a supported version of SignalR on ASP.NET Core, it was a prototype exploration that was discarded prior to the .NET Core 1.0 release. It will not be updated in the future. The 1.0.0 version (currently in Alpha) is the only supported version of SignalR on ASP.NET Core

at least it's not possible to send a message to a specified range of connections ids

This is coming in Preview 1, which should arrive in the next month or so. You can try preview builds using the nightly build instructions here: https://github.com/aspnet/Home#builds

[analogrelay]

Another note: The sample you posted uses project.json, which is from pre-release versions of the .NET Core Tools. You should be using a csproj for all .NET Core development (including .NET Core 1.0/1.1 development). See https://docs.microsoft.com/en-us/dotnet/core/migration/ for more details.

[aleha84]

At the moment of development there was no other solution, but to use 0.2.0. Of cource i will migrate to 2.1 as it will be awailable. Does using csproj instead of project.json make sense? Do you have any advice on the current solution? As I mention, It works stably on the production server, but everything breaks when adding a dapper package.

[aleha84]

Another note: if running from VS as application (not IISExpress), without ANCM in IIS, then all works great, without pending requests. Is this not a sign of having a bug in this module? You can reproduce it in provided example (of cource after migrating to csproj, if this is very important), just running Service project from VS and target MainSite service Url to correct one.

[analogrelay]

Does using csproj instead of project.json make sense?

It is the only officially-supported way to build .NET Core applications (using 1.0, 1.1, 2.0 or 2.1)

At the moment of development there was no other solution, but to use 0.2.0. Of cource i will migrate to 2.1 as it will be awailable.

I get it, but that version is still completely unsupported.

Do you have any advice on the current solution?

I don't have any idea why adding Dapper would do this. It is likely completely unrelated to Dapper.

if running from VS as application (not IISExpress), without ANCM in IIS, then all works great, without pending requests. Is this not a sign of having a bug in this module?

That does indicate it might be an issue with ANCM. We did have some ANCM issues we fixed while working on the new SignalR, so there could be an issue there. I'm not familiar enough with the 0.2.0 code to know what fixes would be applicable and how to back-port them though and as I said earlier, it's not a supported version, so we don't really have the resources to do any work on it.

[aleha84]

Ok, I realized that I shot myself in the leg I can download latest build here: https://github.com/dotnet/cli#installers-and-binaries (https://dotnetcli.blob.core.windows.net/dotnet/Sdk/master/dotnet-sdk-latest-win-x64.exe) But where i can download appropriate WindowsHosting installer?

[analogrelay]

That installer is for .NET Core 2.1 nightlies. We don't currently release nightlies of the Windows Hosting bundle. However, the only thing it adds on top of the Runtime is ANCM. You can install the hosting bundle from 2.0 and then install the 2.1 runtime on top of it and it should run correctly.

[aleha84]

Failed to start project under version 2.1 (nightly build). Installed latest sdk build. dotnet --version outputs: 2.2.0-preview1-007860 dotnet --info outputs: Version : 2.1.0-preview1-26102-03 With csproj:

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.All" Version="2.1.0-preview1-26623" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.SignalR" Version="1.0.0-preview1-27954" />
  </ItemGroup>
</Project>

Exception:

TypeLoadException: Could not load type 'System.Buffers.BufferPool' from assembly 'System.Buffers.Primitives, Version=0.1.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'.

Adding <PackageReference Include="Microsoft.AspNetCore" Version="2.1.0-preview1-27954" /> to csproj, produces another exception:

An unhandled exception was thrown by the application. System.MethodAccessException: Attempt by method 'System.IO.Pipelines.WritableBufferWriter.Write(Byte[])' to access method 'System.Span`1.DangerousGetPinnableReference()' failed.

Replacing version of Microsoft.AspNetCore.All to latest "2.1.0-preview1-27954" produces error:

The specified framework 'Microsoft.AspNetCore.All', version '2.1.0-preview1-27954' was not found.

With .Net Core version 2.0 with <PackageReference Include="Microsoft.AspNetCore.SignalR" Version="1.0.0-alpha2-final" /> the number of simultaneous connections increased to 3, but after it all next requests to service in pending state. Interestingly, it's only I have such a problem, is it really no one else about this did not write an issue?

[aleha84]

I was engaged in further research of the issue. Tried to reproduce it using .Net Framework 4.7. And it outputs exactly the same result. After establishing 2 connections via SignalR IIS hangs absolutely.

I downloaded the sample from https://docs.microsoft.com/en-us/aspnet/signalr/overview/getting-started/tutorial-getting-started-with-signalr-and-mvc (http://code.msdn.microsoft.com/Getting-Started-with-c366b2f3). Hosted it in IIS. Same result. Ok. I took a clean system (win 10 pro), installed IIS there, hosted sample project there. And again same result. But instead of 2 connections it hangs after 9 websocket connections. I tried to change the transport to LongPolling, and server hangs after 6 connections.

IISExpress does not experience such problems. 20+ connections were nadled fine. I do not understand what's going on.