Check session endpoint enabled, but SessionId is missing

[parkinsona]

We have had identityserver in production for about 6 months and just recently (past month and a half) it has started to return the error: "Check session endpoint enabled, but SessionId is missing".

This is only happening in our production environment. Which was moved to a different data centre around the same time the issue started happening. Having said that, we had an increase in usage of the site at the same time too. Both our UAT and PROD environments are load balanced and were moved at the same time.

We haven't been able to replicate it, but we know that it happens when the idsvr cookie is still active, but the idsvr.session cookie is dead. I'm not sure how it could get into this state though.I am yet to determine if its being caused by the "remember me" functionality or not.

This is the setting for cookie options in our IdentityServer host. Expire Time span is 1 hour.

CookieOptions = new Thinktecture.IdentityServer.Core.Configuration.CookieOptions   
                        {
                            SlidingExpiration = true,                           
                            ExpireTimeSpan = Settings.Default.ExpireTimeSpan
                        }

Does this cookie setting cover both the idsvr & idsvr.session cookies that are created? Is it possible that the idsvr cookie is sliding but the idsvr.session cookie is not?

Is there a reason why identityserver doesn't redirect you to the login page when you get into this situation?

Thanks again for your help and support on this product.

[parkinsona]

I should add that i'm running version 1.6.0.0 of identity server.

[parkinsona]

I have further feedback from one of the users experiencing this issue....

They are using IE 11 and this is how they replicated it.

• Go to RP1 and hit login
• Login to identity server and select "Remember me"
• They are logged in and redirected to RP1
• They work in RP1 for a while, then close their browser
• Later in the day they open RP2, and click login
• They get redirected to identity server, but it will return an empty session id.

[brockallen]

Ok, will look into this.

[parkinsona]

I hope you have a break over christmas ;)

[brockallen]

Took a quick look -- it does look like we're issuing a persistent session cookie along with the persistent authentication cookie. It's working here (of course).

So what it sounds like is that your users keep one cookie but not the other. Is that correct?

I'd need some help from your side to help debug this some more.

[parkinsona]

On my machine, I have tested it and it in firefox is shows that it issues a persistent cookie for both session and authentication. On another machine using firefox, it is only showing as a session cookie. But we weren't able to replicate the issue using that.

From what I can see it looks like its keeping the authentication cookie but not the session cookie.

It wouldn't be related to this katana bug? https://katanaproject.codeplex.com/workitem/197

[brockallen]

It wouldn't be related to this katana bug?

Only if you're mixing System.Web cookie code in the same request with OWIN/Katana cookie code. None of the code in IdSvr uses System.Web. Do you have other code in the IdSvr host using System.Web? A http module perhaps?

[parkinsona]

I'm not using System.web cookie code in my host. I do use System.web and other httpmodules for things like logging, and my password reset view (which is used as a partial login)

[brockallen]

Yea, this might be hard to track down then where/what's causing it since neither you or I can repro it ourselves. For all we know, it's the user browser throwing away a cookie, or maybe a load balancer not passing a cookie thru... again, difficult without a repro.

[parkinsona]

Agreed, without the repro I've been struggling. I've only seen it happen once on the users machine, and clearing the cookies and re-logging fixed the issue.

Are there any known issues with load balancers not passing through cookies?

So if we can't track down the cause, should identity server bail at this point and redirect you to the login screen?

[brockallen]

I've heard of LBs dropping cookies when there are too many. But yea, I think that's rare or really just a fluke.

If there's no session at token issuance time, then we can gen one. That might be what we should do, but doing this is somewhat like saying "somewhere else cookies aren't working for some reason, so let's cover it up by doing what should have been done elsewhere".

I don't like that it's just not working right, so I'd like to know why before I put in code to mask other problems,

[brockallen]

But... come to think of it, I might like to move this to the authorization endpoint, since it is an aspect of the OIDC protocol. I'll have to think about it.

[parkinsona]

I agree that covering it up might not be the best option. I am running version 1.6, so its not something that may have been fixed in a newer version? From what I can see, the handling of the session cookie hasn't changed.

The only other thing to mention, is that our sys admin has got the app pool recycling at 3am every night. I wouldn't think this would cause any dramas because our user is only using the site during business hours.

[brockallen]

no, apppool is unrelated. session id is a random value, not related to any keys or anything

[parkinsona]

That's what I thought, just wanted to check.

Here's my startup code, if it helps....

 public void Configuration(IAppBuilder app)
        {
            LogProvider.SetCurrentLogProvider(new DiagnosticsTraceLogProvider());
            //LogProvider.SetCurrentLogProvider(new TraceSourceLogProvider());

            // uncomment to enable HSTS headers for the host
            // see: https://developer.mozilla.org/en-US/docs/Web/Security/HTTP_strict_transport_security
            //app.UseHsts();

            app.Map("/core", coreApp =>
                {

                    LoginPageLink resetLink = new LoginPageLink() { Href = Settings.Default.PasswordResetLink, Text="Forgot Password?" };

                    var options = new IdentityServerOptions
                    {

                        IssuerUri = "https://mysite.com.au",                       
                        SiteName = "MySite IdentityServer",

                        PublicOrigin = Settings.Default.PublicHostName,
                        SigningCertificate = Cert.Load(),
                        Factory = Factory.Configure("MembershipReboot"), // This is the name of the connection string
                        CorsPolicy = CorsPolicy.AllowAll,

                        RequireSsl = true,
                        AuthenticationOptions = new AuthenticationOptions
                        {
                           // IdentityProviders = ConfigureIdentityProviders,
                            LoginPageLinks = new LoginPageLink[] { resetLink },
                        CookieOptions = new Thinktecture.IdentityServer.Core.Configuration.CookieOptions   
                        {
                            SlidingExpiration = true,                           
                            ExpireTimeSpan = Settings.Default.ExpireTimeSpan
                        }
                        }

                    };

                    coreApp.UseIdentityServer(options);
                });

            // only for showing the getting started index page
            app.UseStaticFiles();

            //Use Log4Net to log events
            XmlConfigurator.Configure();            
            LogProvider.SetCurrentLogProvider(new Log4NetLogProvider());

        }

[parkinsona]

Hi Brock, did you have any more thoughts on this issue?

[brockallen]

No, not really. @leastprivilege and I are meeting next week, so we can discuss then.

[parkinsona]

Hey Brock,

I have some more info regarding this issue. We had another user have the issue today (11/1). This time it was for another relying party. After speaking with 2 users of this application, 1 used "Remember Me" and the other didn't. The one who didn't use "Remember Me" was able to log in today, the one who did couldn't log in.

Both were using the same RP, IE 11 and were accessing the site at the same time. Both could log in on (8/11) This fits with our previous case, where the only user having the issue was one who chose the "Remember me" option.

Unfortunately, they cleared their cookies before I could investigate the issue. So that's all I have.

[brockallen]

I tried to repro it again here locally with IE11 and I can't get the behavior you described (when I check "remember my login").

[brockallen]

Also, just to clarify; you said:

We have had identityserver in production for about 6 months and just recently (past month and a half) it has started to return the error: "Check session endpoint enabled, but SessionId is missing".

But it's not preventing users from logging in, right? It's just not issuing a session_state value in the response, correct? This is just showing up in the logs, right?

[brockallen]

Also, are you doing any custom login screen where you're taking over issuing the login cookie?

[parkinsona]

Hi Brock,

It is preventing users from logging in. It's as though they browser loses the session_state cookie after a certain period of time. To allow them to log in again, they've been clearing their cookies and starting authentication again.

I have custom styles for the login screen and I have a custom partial sign in page that will prompt users to reset their passwords when they have expired.

[brockallen]

Where or how exactly does it prevent them from logging in? Does IDSvr throw an exception?

[parkinsona]

When the RP redirects to idsvr, idsvr throws an exception.

Here's what's happening in the log prior:

2016-01-25 15:37:47,884 [56] INFO  Thinktecture.IdentityServer.Core.Endpoints.AuthorizeEndpointController - Start authorize request

2016-01-25 15:37:47,884 [56] INFO  Thinktecture.IdentityServer.Core.Validation.AuthorizeRequestValidator - Start authorize request protocol validation

2016-01-25 15:37:47,900 [56] ERROR Thinktecture.IdentityServer.Core.Validation.AuthorizeRequestValidator - Check session endpoint enabled, but SessionId is missing
 {
  "ClientId": "mysite_implicitclient",
  "ClientName": "mysite",
  "RedirectUri": "https://mysite.com.au/",
  "AllowedRedirectUris": [
    "https://mysite.com.au/"
  ],
  "SubjectId": "64fbc3c2-86bb-41b8-b141-f97540c7aa18",
  "ResponseType": "id_token",
  "ResponseMode": "form_post",
  "Flow": "Implicit",
  "RequestedScopes": "openid email profile myscope",
  "State": "OpenIdConnect.AuthenticationProperties=g5XIMum2EI1Q3LvrgvwVWmCVqTM8QQzhQAlYseGUsnoFS9Ogs7cj5NrXnlmjpQEgOMJy7f4Re5cugVM2_aavm_zgof5-78vo6Uc42OvgRGORJz3Rqx8ATM8-_BvkRdJdZdr5sD_lDZi4v3Zz2DwPvgAqPtEc5uDCyckfkP5RU1Vf3moAiCQyWhQMnLgSD3YlcTWX31vsIuvFYDSSSHyzZ_PLZKRyQFOaXWd619dQU14",
  "Nonce": "635892934679230492.MzhmNGQ2Y2UtNDJiNS00M2E4LWFiMWMtODE4NjdiNmU5Y2EwNThiMjZmODgtNmQ5ZS00MTZlLTg4ODMtMjQ2ZWM5NjkxMTli",
  "Raw": {
    "client_id": "mysite_implicitclient",
    "redirect_uri": "https://mysite.com.au/",
    "response_mode": "form_post",
    "response_type": "id_token",
    "scope": "openid email profile myscope",
    "state": "OpenIdConnect.AuthenticationProperties=g5XIMum2EI1Q3LvrgvwVWmCVqTM8QQzhQAlYseGUsnoFS9Ogs7cj5NrXnlmjpQEgOMJy7f4Re5cugVM2_aavm_zgof5-78vo6Uc42OvgRGORJz3Rqx8ATM8-_BvkRdJdZdr5sD_lDZi4v3Zz2DwPvgAqPtEc5uDCyckfkP5RU1Vf3moAiCQyWhQMnLgSD3YlcTWX31vsIuvFYDSSSHyzZ_PLZKRyQFOaXWd619dQU14",
    "nonce": "635892934679230492.MzhmNGQ2Y2UtNDJiNS00M2E4LWFiMWMtODE4NjdiNmU5Y2EwNThiMjZmODgtNmQ5ZS00MTZlLTg4ODMtMjQ2ZWM5NjkxMTli"
  }
}

I'll add some screenshots in another message.

[brockallen]

What version of IdSvr? It looks like 1.x something?

[parkinsona]

I'm running 1.6.0.0 in PROD. We have version 2.3 in DEV, which I'm waiting on some internal approvals to deploy through to TEST and UAT to see if we can replicate in the latest version.

[brockallen]

When the RP redirects to idsvr, idsvr throws an exception.

That's a log error, not a runtime exception. I'm still unclear how users are prevented from logging in...

But anyway, perhaps 2.3 is your best best on this -- is it possible to test if your 2.3 dev is still having this issue?

[parkinsona]

Sorry, I had that wrong. It's not idsvr with the exception, its the RP.
The RP checks for the sessionState claim, and throws an exception when it doesn't find it. The RP is using the sessionState claim for single sign out of the RPs.

[brockallen]

So again, I'm not sure what we should do with this. Is there any way you can reproduce this with our samples?

[parkinsona]

Unfortunately I've not been able to replicate it with samples. I know we had discussed adding something to handle the scenario when no session cookie was present, but you had to think more about that.

If worst comes to worst, I can turn off the "remember me" option.In the meantime I'm happy to wait this out and see if it continues to happen in PROD after I update to the latest release of idsvr. This might be a few weeks away with my current release schedules.

[brockallen]

Ideally you would be able to repro with our samples. If not then perhaps you can slowly add your custom code to our samples to determine the trigger point so we understand what's causing it.

-Brock

On Jan 31, 2016, at 5:34 PM, parkinsona notifications@github.com wrote:

Unfortunately I've not been able to replicate it with samples. I know we had discussed adding something to handle the scenario when no session cookie was present, but you had to think more about that.

If worst comes to worst, I can turn off the "remember me" option.In the meantime I'm happy to wait this out and see if it continues to happen in PROD after I update to the latest release of idsvr. This might be a few weeks away with my current release schedules.

— Reply to this email directly or view it on GitHub.

[parkinsona]

I can't even repro with my own code. It seems to only be happening for a few users, and it only happens about once a month.Unfortunately those users aren't on site, so often they have cleared cookies before I get the chance to see what's happening in their browser. I've instructed them to let me know when it happens.

[brockallen]

Ok, got it. Well, keep us posted if you learn more.

-Brock

On Jan 31, 2016, at 5:52 PM, parkinsona notifications@github.com wrote:

I can't even repro with my own code. It seems to only be happening for a few users, and it only happens about once a month.Unfortunately those users aren't on site, so often they have cleared cookies before I get the chance to see what's happening in their browser. I've instructed them to let me know when it happens.

— Reply to this email directly or view it on GitHub.

[parkinsona]

Absolutely :+1:

Thanks again for your help looking at this.

[parkinsona]

We had this issue this morning with an external customer. Unfortunately we couldn't troubleshoot with them because they just wanted to login again.

However, I can confirm that it is definitely the remember me option in IE.

We noticed a few extra things this time round. The user had changed their password just last night, so the cookie hadn't expired. Your previous comments mentioned using System.Web cookie, which I thought maybe the partial login to change password might be causing it, although this doesn't use cookies. I was also able to disprove this theory because the last time we had this issue the user hadn't changed their password for quite some time.

I thought I'd have another look for cookies and I did find this in the web.config. I must have dismissed it first time because it was in config.

  <system.web>
     ....
    <httpCookies httpOnlyCookies="true" requireSSL="true" />
    ....
  </system.web>

Interestingly.....the idsvr.session cookie is the only cookie being issued that is not HttpOnly. I'm not sure if that could be the cause?

[parkinsona]

As an FYI, I tried removing the httpCookies setting from my web.config and it produced the same results.

[parkinsona]

One of our internal users had the issue yesterday and I was able to hop on their machine and have a look around. Attached is the xml from IE's F12, which can be loaded into fiddler.

From what I can tell, the idsvr.session cookie just isn't getting passed through from the RP to idsvr. NetworkData_20160208.zip

[brockallen]

Ok, took a look. The log actually starts a bit too late -- it shows after the login has taken place, so we don't see the response from the login page where the session id should have been issued.

The one thing cookie related that looks odd to me is this cookie:

<cookie>
 <name>_ga</name>
 <value>GA1.3.1517871657.1451868525</value>
</cookie>

What this tells me is that there's something in your web server or network that's adding more cookies. Perhaps it's somehow involved with the session cookie not making it back to the browser?

[parkinsona]

I thought the same about the log being too late. But that's when the issue occurs. When they initially login to idsvr they're all good and can work and there is a session cookie. I know that the session cookie exists initially because the RPs implement checkSession iframe for single signout. Without the idsvr.session cookie they would be logged out automatically after 30 seconds of being on the RP. At least that's my understanding of how that works.

The issue happens a few days after they have been using the site for a few days. In the morning they'll open their browser, go to the RP and then it will give you the log I sent through. To me, it seems like the browser might be ditching the cookie.

I thought the _ga cookie was odd as well. It supposedly for Google Analytics. However, to my knowledge, none of our sites use Google Analytics. I know for certain that idsvr doesn't use GA. I will need to confirm this in the other RPs.

[brockallen]

Come to think of it, the other cookie i don't see is the antixsrf cookie.

[brockallen]

Maybe it's related to the fact there are dots in those cookies....?

[parkinsona]

That's a good point about the antixsrf. I just looked at my current session in firefox and I had idsvr and idsvr.session cookies, but not the antixsrf cookie. I always thought it was short lived and only used for the initial login, which is why it was gone all the time.

However, if its supposed to be there. Perhaps, there's something else at play.

[brockallen]

If they just had logged in, then I'd expect to see the antixsrf cookie.

[parkinsona]

As the users affected are selecting "Remember Me", they could have logged in a day ago, or 3 weeks ago.

[parkinsona]

Hey Brock,

I can confirm that none of our RPs are using Google analytics, so I'm not sure why that's being included. Perhaps something in the browser is keeping that ga cookie around.

I think you have a point about both the antixsrf and session cookies both having a "." has some merit. I checked on another pc we've got setup to catch this error, and even after several days all 3 cookies are there (idsvr, idsvr.session, idsvr.xsrf).

[brockallen]

The other thing I was wondering is if these users have any browser plugins that might be affecting things.... just another thought.

[parkinsona]

I've requested a list of plugins used by the user. I'm waiting on a response. Will keep you posted.

[brockallen]

Sorry this is taking so long to resolve, but i'm 99% sure there's something environmental.

[parkinsona]

It's all good. It's not your fault its taking so long. The main person who has the issue only works a couple of days a week, so it takes a while to get responses on my end.

I'm pretty sure its environmental as well.

[parkinsona]

Hey Brock,

I got the feedback about the add-ons in IE. They were:

• Shockwave Flash Object
• Send to OneNote
• OneNote Linked Notes

There were also some disabled add-ons:

• Skype for Business Browser Hel....
• Skype for Business Click to Call
• Symantec Vulnerability Protection

hmmmmmm I wonder... When we were working on our bulk upload, one of the other developers had an issue with identity server and skype. Let me chase it up with him and get back to you. Could be a while because he only works 1 day a week :S