search

aspnet / AspNetWebStack

ASP.NET MVC 5.x, Web API 2.x, and Web Pages 3.x (not ASP.NET Core)
Other
852 stars 354 forks source link

The anti-forgery cookie is not present #123

Closed superainovacoes closed 6 years ago

superainovacoes commented 6 years ago

Hi, i have a website build on asp net mvc 5. Currently the website works fine but when google is crawling it i can see erros about : The required anti-forgery cookie "__RequestVerificationToken" is not present.

The website loads normaly, then we make a jquery post request to server to get some data. This post uses the "@Html.AntiForgeryToken()" and the controller action has "[ValidateAntiForgeryToken]" When a normal user hit the site, like me, i cant reproduce this error. Only happens when google craw it

Here is some report about the google craw request:

{
    "Exception": {
        "Message": "The required anti-forgery cookie \"__RequestVerificationToken\" is not present.",
        "Data": {},
        "Source": "System.Web.WebPages",
        "StackTrace": "   at System.Web.Helpers.AntiXsrf.TokenValidator.ValidateTokens(HttpContextBase httpContext, IIdentity identity, AntiForgeryToken sessionToken, AntiForgeryToken fieldToken)\r\n   at System.Web.Helpers.AntiXsrf.AntiForgeryWorker.Validate(HttpContextBase httpContext)\r\n   at System.Web.Mvc.ControllerActionInvoker.InvokeAuthorizationFilters(ControllerContext controllerContext, IList`1 filters, ActionDescriptor actionDescriptor)\r\n   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass21.<BeginInvokeAction>b__19(AsyncCallback asyncCallback, Object asyncState)",
        "TargetSite": {
            "Name": "ValidateTokens",
            "AssemblyName": "System.Web.WebPages, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35",
            "ClassName": "System.Web.Helpers.AntiXsrf.TokenValidator",
            "Signature": "Void ValidateTokens(System.Web.HttpContextBase, System.Security.Principal.IIdentity, System.Web.Helpers.AntiXsrf.AntiForgeryToken, System.Web.Helpers.AntiXsrf.AntiForgeryToken)",
            "Signature2": "System.Void ValidateTokens(System.Web.HttpContextBase, System.Security.Principal.IIdentity, System.Web.Helpers.AntiXsrf.AntiForgeryToken, System.Web.Helpers.AntiXsrf.AntiForgeryToken)",
            "MemberType": 8,
            "GenericArguments": null
        },
        "BaseException": {
            "Message": "The required anti-forgery cookie \"__RequestVerificationToken\" is not present.",
            "Data": {},
            "Source": "System.Web.WebPages",
            "StackTrace": "   at System.Web.Helpers.AntiXsrf.TokenValidator.ValidateTokens(HttpContextBase httpContext, IIdentity identity, AntiForgeryToken sessionToken, AntiForgeryToken fieldToken)\r\n   at System.Web.Helpers.AntiXsrf.AntiForgeryWorker.Validate(HttpContextBase httpContext)\r\n   at System.Web.Mvc.ControllerActionInvoker.InvokeAuthorizationFilters(ControllerContext controllerContext, IList`1 filters, ActionDescriptor actionDescriptor)\r\n   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass21.<BeginInvokeAction>b__19(AsyncCallback asyncCallback, Object asyncState)",
            "TargetSite": {
                "Name": "ValidateTokens",
                "AssemblyName": "System.Web.WebPages, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35",
                "ClassName": "System.Web.Helpers.AntiXsrf.TokenValidator",
                "Signature": "Void ValidateTokens(System.Web.HttpContextBase, System.Security.Principal.IIdentity, System.Web.Helpers.AntiXsrf.AntiForgeryToken, System.Web.Helpers.AntiXsrf.AntiForgeryToken)",
                "Signature2": "System.Void ValidateTokens(System.Web.HttpContextBase, System.Security.Principal.IIdentity, System.Web.Helpers.AntiXsrf.AntiForgeryToken, System.Web.Helpers.AntiXsrf.AntiForgeryToken)",
                "MemberType": 8,
                "GenericArguments": null
            }
        }
    },
    "Url": "https://www.mysite.com.br/some-post-request",
    "UrlReferrer": "https://www.mysite.com.br/some-page",
    "UserAgent": "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)",
    "UserHostAddress": "66.249.69.139",
    "UserHostName": "66.249.69.139",
    "UserLanguages": ["en-US"],
    "Cookies": [{
        "key": "__RequestVerificationToken_L3JlZGVwYXM1",
        "value": {
            "Name": "__RequestVerificationToken_L3JlZGVwYXM1",
            "Path": "/",
            "Secure": false,
            "Shareable": false,
            "HttpOnly": false,
            "Expires": "0001-01-01T00:00:00",
            "Value": "HOCtP-6X2gBEXpo1NDFxexJyDE-hWi5EbbNe6Tc5v0pwV0MbOAtYX_cscKFOtCOTPneQ1kqjeGTB8OyEhD4gIBRfYu41",
            "HasKeys": false,
            "Values": [null]
        }
    }],
    "Form": [{
        "key": "__RequestVerificationToken",
        "value": "9lyKye9Ss8DLvKB4fFhjgCuvWVirFP65LpWeWpOk1HiReI_KZ2m8weIXt_CXE9k3PSmv7X1zjww5HDcz4QaUXZj40dw1"
    ],
    "QueryString": [],
    "Headers": [{
        "key": "Cookie",
        "value": "__RequestVerificationToken_L3JlZGVwYXM1=HOCtP-6X2gBEXpo1NDFxexJyDE-hWi5EbbNe6Tc5v0pwV0MbOAtYX_cscKFOtCOTPneQ1kqjeGTB8OyEhD4gIBRfYu41"
    }, {
        "key": "From",
        "value": "googlebot(at)googlebot.com"
    }, {
        "key": "Host",
        "value": "www.mysite.com.br"
    }, {
        "key": "Referer",
        "value": "https://www.mysite.com.br/some-page"
    }, {
        "key": "User-Agent",
        "value": "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
    }, {
        "key": "X-Requested-With",
        "value": "XMLHttpRequest"
    }, {
        "key": "__RequestVerificationToken",
        "value": "XF3W5OogV-cLsjJb4GgLRL2wkD-4A6PKHYMR8tllJDvuTPeLxjIwyxxsCHYuZBlxhlYifRem9JsZRruDIvVndYUWwpQ1"
    }],
    "HttpMethod": "POST"
}

Note that the error is validating the "cookie" token (i am not setting to use cookie), note the value from request post. Can i disable the cookie validation?

My "Application_Start" has this config:

            AntiForgeryConfig.AdditionalDataProvider = null;
            AntiForgeryConfig.CookieName = "__RequestVerificationToken";
            AntiForgeryConfig.RequireSsl = false;
            AntiForgeryConfig.SuppressIdentityHeuristicChecks = false;
            AntiForgeryConfig.SuppressXFrameOptionsHeader = false;
            AntiForgeryConfig.UniqueClaimTypeIdentifier = null;

package.config

  <package id="Microsoft.AspNet.Mvc" version="5.2.3" targetFramework="net452" />
  <package id="Microsoft.AspNet.Razor" version="3.2.3" targetFramework="net452" />
  <package id="Microsoft.AspNet.WebPages" version="3.2.3" targetFramework="net452" />
  <package id="Microsoft.Extensions.Configuration" version="1.1.1" targetFramework="net452" />
  <package id="Microsoft.Extensions.Configuration.Abstractions" version="1.1.1" targetFramework="net452" />
  <package id="Microsoft.Extensions.Primitives" version="1.1.0" targetFramework="net452" />
  <package id="Microsoft.Net.Compilers" version="1.3.2" targetFramework="net452" developmentDependency="true" />
  <package id="Microsoft.NETCore.Platforms" version="2.0.1" targetFramework="net452" />
  <package id="Microsoft.Web.Infrastructure" version="1.0.0.0" targetFramework="net452" />
mkArtakMSFT commented 6 years ago

@dougbu, do you have any thoughts regarding this?

dougbu commented 6 years ago

An antiforgery token is present but has an unexpected name. Suspect the configuration was changed after the crawler got that cookie. Or, another site behind the same hostname set its cookie.

Recommend not setting CookieName. Unique cookie names are useful to avoid conflicts between ASP.NET services on the same host.

Also recommend AntiForgeryConfig.RequireSsl = true if the site exclusively uses HTTPS.

Finally, it's odd the Google crawler POSTs anywhere. Might want to disable antiforgery entirely for the crawler's UserAgent. For example, use [ValidateAntiForgeryToken(WrappingAction)] instead of [ValidateAntiForgeryToken] and write WrappingAction to only call AntiForgery.Validate for other clients.

superainovacoes commented 6 years ago

We have the same app aplications published to multiple virtual directorys. They all use the same machinekey and cookie auth based on formsauthentication. example: site.com/name1 site.com/name2 site.com/name3 the differente between the applications is a value key in appsettins to load particular configuration

I'll try the RequireSsl and the WrappingAction and i'll report later.

superainovacoes commented 6 years ago

Hi, logging this error we have noticed that the problem is with mobile browsers (by checking user-agents) Examples:

Mozilla/5.0 (Linux; Android 7.0; Moto G (5) Plus Build/NPN25.137-92) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.137 Mobile Safari/537.36
Mozilla/5.0 (Linux; Android 4.2.2; ALCATEL ONE TOUCH 7040E Build/JDQ39) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.123 Mobile Safari/537.36
mkArtakMSFT commented 6 years ago

@superainovacoes, did you get a chance to try out @dougbu's suggestion with the wrapping action?

superainovacoes commented 6 years ago

Which version is this included? Currently on 5.2.3.0 this option is unavailable. image

dougbu commented 6 years ago

My apologies @superainovacoes, the [ValidateAntiForgeryToken(Action)] constructor is unfortunately internal. But, that point was just about the oddity of a web crawler POSTing to your site. What happens when you use the default AntiForgeryConfig? And, when you only set RequireSsl = true?

Separately, your comment about mobile clients is confusing. Your original post said problems only happened when Google crawls the site.

superainovacoes commented 6 years ago

Before making a issue here, i wasnt setting any values for the AntiForgeryConfig. After searching i found the AntiForgeryConfig, then config for CookieName gave no results. I'm planning going production with RequireSsl=true today.

About the mobile just dont know :( , i am receiving some logs about it. When i navigate the site on my mobile i dont have errors, seens to be "random", these mobile reports shows anothers cookies (google analytics for example) but dont contains the __RequestVerificationToken.

But since the page posts the __RequestVerificationToken it should check for cookies either?

Anothers mobile useragents that give errors:

Mozilla/5.0 (Linux; Android 7.0; Moto G (4) Build/NPJS25.93-14-13) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.137 Mobile Safari/537.36
Mozilla/5.0 (Linux; Android 6.0.1; SAMSUNG SM-G570M Build/MMB29K) AppleWebKit/537.36 (KHTML, like Gecko) SamsungBrowser/6.2 Chrome/56.0.2924.87 Mobile Safari/537.36
Mozilla/5.0 (Linux; Android 5.1.1; SM-J120H Build/LMY47V) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.137 Mobile Safari/537.36
dougbu commented 6 years ago

With the default setup, two tokens are required -- one in a cookie and the other as a form field. The two tokens must be consistent with each other or the form submission will fail. But, the errors you're describing are rare because the HTML response containing the form also contains the two tokens.

I suspect either

  1. The client has switched to HTTPS after receiving the tokens over HTTP. If the client starts on HTTPS, this shouldn't happen.
  2. The client is submitting manually-created data mimicking a correct payload but using a __RequestVerificationToken value from another session.
superainovacoes commented 6 years ago

@dougbu

  1. We always load the site under HTTPS.
  2. Some request are made using jquery. We read the hidden input value from the page and submit it. We dont change anything on cookie about the __RequestVerificationToken
superainovacoes commented 6 years ago

Hi, feedback for RequireSsl=true did not work. The "error" continues. Again, the log show only mobile useragents :(

The case is the same, page opens and post some data to another url to grab information, this post returns the error "The required anti-forgery cookie "__RequestVerificationToken" is not present."

Google crawler error are almost none (i've included the ajax url to robots to disallow request)

dougbu commented 6 years ago

@superainovacoes thank-you for filing this issue. But, in order for us to investigate this issue, please provide a minimal repro project that illustrates the problem, preferably hosted in a GitHub repo.

FYI I've run a number of scenarios involving mobile clients and [ValidateAntiForgeryToken]. I have not seen any problems.

dougbu commented 6 years ago

Clearing the investigate label in lieu of more information.

dougbu commented 6 years ago

Thank you for your feedback. We're closing this issue as no updates have been provided in a timely manner and we have been unable to reproduce the issue. If you have more details and are encountering this issue please add a new reply and re-open the issue.

superainovacoes commented 6 years ago

Sorry for the delay, i'll create a sample project.

Scenario:

user open the site
- mvc generetes a forgery token
user still in the same page
- meanwhile iis is recycled (manually or automatically)
user do some action that makes a post to mvc with the initial token

Question: the request still valid or it will be broken? iis is using "machineKey"

dougbu commented 6 years ago

Question: the request still valid or it will be broken?

The machine key used to encrypt the tokens changes whenever the application starts. This means the tokens cannot be decreypted after an application or IIS restart. This can be avoided by setting a static machine key in web.config.

In addition, the tokens may not be valid if a user was authenticated when the tokens were generated. This depends on the authentication method used and won't happen if the authentication is not relevant in your scenario.

superainovacoes commented 6 years ago

Ok, so I have a machinekey configured for my app pool inside the web.config. So when/if the pool restarts i should not get this error, correct? But still the exception is raising, the token has max time to be valid?

Maybe the user has the page open for days, then he does some action that use the token. This can be expired based beeing to "old"?

I ask this because the errors still only in mobile, users can open the page change app and come back after hours/days then the token is invalid.

--- edit

web.config

    <authentication mode="Forms">
      <forms name=".AUTHCOOKIE" />
    </authentication>
    <machineKey decryptionKey="***" validation="SHA1" validationKey="***" />

in .net we use persist the loggedin client

FormsAuthentication.SetAuthCookie

but the error happns even if the user is not loggedin

superainovacoes commented 6 years ago

@dougbu Full log report, cookie and form token are not equal.

{
    "Exception": {
        "Message": "The anti-forgery cookie token and form field token do not match.",
        "Data": {},
        "Source": "System.Web.WebPages",
        "StackTrace": "   at System.Web.Helpers.AntiXsrf.TokenValidator.ValidateTokens(HttpContextBase httpContext, IIdentity identity, AntiForgeryToken sessionToken, AntiForgeryToken fieldToken)\r\n   at System.Web.Helpers.AntiXsrf.AntiForgeryWorker.Validate(HttpContextBase httpContext)\r\n   at System.Web.Mvc.ControllerActionInvoker.InvokeAuthorizationFilters(ControllerContext controllerContext, IList`1 filters, ActionDescriptor actionDescriptor)\r\n   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__0(AsyncCallback asyncCallback, Object asyncState)",
        "TargetSite": {
            "Name": "ValidateTokens",
            "AssemblyName": "System.Web.WebPages, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35",
            "ClassName": "System.Web.Helpers.AntiXsrf.TokenValidator",
            "Signature": "Void ValidateTokens(System.Web.HttpContextBase, System.Security.Principal.IIdentity, System.Web.Helpers.AntiXsrf.AntiForgeryToken, System.Web.Helpers.AntiXsrf.AntiForgeryToken)",
            "Signature2": "System.Void ValidateTokens(System.Web.HttpContextBase, System.Security.Principal.IIdentity, System.Web.Helpers.AntiXsrf.AntiForgeryToken, System.Web.Helpers.AntiXsrf.AntiForgeryToken)",
            "MemberType": 8,
            "GenericArguments": null
        },
        "BaseException": {
            "Message": "The anti-forgery cookie token and form field token do not match.",
            "Data": {},
            "Source": "System.Web.WebPages",
            "StackTrace": "   at System.Web.Helpers.AntiXsrf.TokenValidator.ValidateTokens(HttpContextBase httpContext, IIdentity identity, AntiForgeryToken sessionToken, AntiForgeryToken fieldToken)\r\n   at System.Web.Helpers.AntiXsrf.AntiForgeryWorker.Validate(HttpContextBase httpContext)\r\n   at System.Web.Mvc.ControllerActionInvoker.InvokeAuthorizationFilters(ControllerContext controllerContext, IList`1 filters, ActionDescriptor actionDescriptor)\r\n   at System.Web.Mvc.Async.AsyncControllerActionInvoker.<>c__DisplayClass3_1.<BeginInvokeAction>b__0(AsyncCallback asyncCallback, Object asyncState)",
            "TargetSite": {
                "Name": "ValidateTokens",
                "AssemblyName": "System.Web.WebPages, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35",
                "ClassName": "System.Web.Helpers.AntiXsrf.TokenValidator",
                "Signature": "Void ValidateTokens(System.Web.HttpContextBase, System.Security.Principal.IIdentity, System.Web.Helpers.AntiXsrf.AntiForgeryToken, System.Web.Helpers.AntiXsrf.AntiForgeryToken)",
                "Signature2": "System.Void ValidateTokens(System.Web.HttpContextBase, System.Security.Principal.IIdentity, System.Web.Helpers.AntiXsrf.AntiForgeryToken, System.Web.Helpers.AntiXsrf.AntiForgeryToken)",
                "MemberType": 8,
                "GenericArguments": null
            }
        }
    },
    "Url": "https://www.???.com:443/stats/produto/view",
    "UrlReferrer": "https://www.???.com/ofertas",
    "UserAgent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36",
    "UserHostAddress": "???",
    "UserHostName": "???",
    "UserLanguages": ["pt-BR", "pt;q=0.9", "en-US;q=0.8", "en;q=0.7"],
    "Cookies": [{
        "key": ".AUTHCOOKIE",
        "value": {
            "Name": ".AUTHCOOKIE",
            "Path": "/",
            "Secure": false,
            "Shareable": false,
            "HttpOnly": false,
            "Expires": "0001-01-01T00:00:00",
            "Value": "5360C64595D???",
            "HasKeys": false,
            "Values": [null]
        }
    }, {
        "key": "__RequestVerificationToken",
        "value": {
            "Name": "__RequestVerificationToken",
            "Path": "/",
            "Secure": false,
            "Shareable": false,
            "HttpOnly": false,
            "Expires": "0001-01-01T00:00:00",
            "Value": "bJR8K9bw8cYBrtixVk5GrlsuHoRwsmaAjNkVp1xcIkHAnkSkQjxCfBM2rkcR16V7kGLXwT2g8yoBhGWoJbF1tGUQZg81",
            "HasKeys": false,
            "Values": [null]
        }
    }],
    "Form": [{
        "key": "__RequestVerificationToken",
        "value": "ZD7JKwVNSpXkpND71qPYbDrF0M5jupGoGV6eou8rn4HA4PAQPpskQjNTsgaQ7KV6H3YmWbmMRAk0vLaXkOGlafkAy2bEHFl8TovAa167U2P1wqst0"
    }, {
        "key": "id",
        "value": "69434"
    }],
    "QueryString": [],
    "Headers": [{
        "key": "Cache-Control",
        "value": "no-cache"
    }, {
        "key": "Connection",
        "value": "close"
    }, {
        "key": "Content-Length",
        "value": "149"
    }, {
        "key": "Content-Type",
        "value": "application/x-www-form-urlencoded; charset=UTF-8"
    }, {
        "key": "Accept",
        "value": "*/*"
    }, {
        "key": "Accept-Encoding",
        "value": "gzip, deflate, br"
    }, {
        "key": "Accept-Language",
        "value": "pt-BR,pt;q=0.9,en-US;q=0.8,en;q=0.7"
    }, {
        "key": "Cookie",
        "value": ".AUTHCOOKIE=5360C64595D???; __RequestVerificationToken=bJR8K9bw8cYBrtixVk5GrlsuHoRwsmaAjNkVp1xcIkHAnkSkQjxCfBM2rkcR16V7kGLXwT2g8yoBhGWoJbF1tGUQZg81;"
    }, {
        "key": "Host",
        "value": "www.???.com"
    }, {
        "key": "Referer",
        "value": "https://www.???.com/ofertas"
    }, {
        "key": "User-Agent",
        "value": "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36"
    }, {
        "key": "origin",
        "value": "https://www.???.com"
    }, {
        "key": "x-requested-with",
        "value": "XMLHttpRequest"
    }, {
        "key": "__requestverificationtoken",
        "value": "ZD7JKwVNSpXkpND71qPYbDrF0M5jupGoGV6eou8rn4HA4PAQPpskQjNTsgaQ7KV6H3YmWbmMRAk0vLaXkOGlafkAy2bEHFl8TovAa167U2P1wqst0"
    }],
    "HttpMethod": "POST",
    "ClientCertificate": []
}

I've added "???" to hide the site information.

dougbu commented 6 years ago

Hi, it looks like you are posting on a closed issue/PR/commit!

In particular, the symptoms you're now describing relate to token expiration and not machine key changes.

Details

Users who have been idle for a while should refresh a form before submitting it.

We're very likely to lose track of your bug/feedback/question unless you:

  1. Open a new issue
  2. Explain very clearly what you need help with
  3. If you think you have found a bug, include detailed repro steps so that we can investigate the problem
fabiomaulo commented 3 years ago

I know this is a closed issue and it is too late but... I´m working on a web site mostly used from mobile and after some investigation about users, IP etc. I have one hypothesis: The anti-forgery cookie expires with the HTTP session and mobile devices can close/change the session for various scenarios (tab, windows, change app... I don't know exactly). Perhaps this is the reason because the exception seems so random.