Set AccessToken Validation on a .net wepapi 2 (not core) with Identity server 4

[brunosantos]

I know how to setup IdentityServer 4 Authentication in .Net core. That is: using the extensions defined in IdentityServer4.AccessTokenValidation. And I would set it up in my startup class like so:

    app.UseIdentityServerAuthentication(new IdentityServerAuthenticationOptions
    {
        Authority = "http://localhost:5000",
        RequireHttpsMetadata = false,

        ApiName = "webapi"
    });

The problem is that now I need to make authenticated requests to a .net 4.6 web api2 (not core). And the same package doesn't work for that.

According to this question all I have to do is to use the same package that was used for Identity server 3:IdentityServer3.AccessTokenValidation.

But After trying it out all I get is 401 when making requests to the web api. And I don't know how to wire authentication events to understand the reason behind it. Here is my configuration:

Api Startup.cs:

    app.UseIdentityServerBearerTokenAuthentication(new IdentityServerBearerTokenAuthenticationOptions
    {
        Authority = "http://localhost:5000",
        RequiredScopes = new[] { "webapi" },     
    });

Client Startup.cs:

            app.UseOpenIdConnectAuthentication(new OpenIdConnectAuthenticationOptions
            {
                AuthenticationType = "oidc",
                SignInAsAuthenticationType = "Cookies",
                Authority = "http://localhost:5000",
                RedirectUri = "http://localhost:3954/signin-oidc",
                ClientId = "MvcClient",
                Scope = "openid profile webapi offline_access",
                ResponseType = "code id_token",
                ClientSecret = "secret",
                UseTokenLifetime = false,
                TokenValidationParameters = new TokenValidationParameters
                {
                    ValidateIssuer = false,
                },
});

TestController in the Client project:

    var tokenClient = new TokenClient("http://localhost:5000/connect/token", "MvcClient", "secret");
    var tokenResponse = await tokenClient.RequestClientCredentialsAsync("webapi");

    var client = new HttpClient();
    client.SetBearerToken(tokenResponse.AccessToken);
    var content = await client.GetStringAsync("http://localhost:5004/api/identity");

I successfully get an access token here. But get a 401 when making the request to api/identity.

Here is the Config in the IDP:

new ApiResource("webapi", "My API")
[...]
                new Client
                {
                    ClientId = "MvcClient",
                    ClientName = "MVC Client",
                    AllowedGrantTypes = GrantTypes.HybridAndClientCredentials,

                    RequireConsent = true,

                    ClientSecrets =
                    {
                        new Secret("secret".Sha256())
                    },

                    RedirectUris = { "http://localhost:3954/signin-oidc" },
                    PostLogoutRedirectUris = { "http://localhost:3954/signout-callback-oidc" },

                    AllowedScopes =
                    {
                        IdentityServerConstants.StandardScopes.OpenId,
                        IdentityServerConstants.StandardScopes.Profile,
                        "netcoremvcapi",
                        "webapi"
                    },
                    AllowOfflineAccess = true,
                }

Any idea why this might be failing? Am I making the wrong assumption that I can use IdentityServer3.AccessTokenValidation to validate the token?