Scenario is not replaying set cookies

[KallDrexx]

I'm currently running 1.6.0-23.

I am assuming I am just misunderstanding something but I can't get auth cookies that the server is setting to be replayed by subsequent requests later in the scenario. When a user successfully logs into our api we return an HttpOnly cookie called Fuze. So I have the following configuration:

config:
  payload:
    path: "logins.csv"
    fields:
      - "username"
      - "password"
  phases:
    - duration: 300
      arrivalRate: 1
scenarios:
  - name: "Test"
    flow:
      - post:
          url: "/login"
          json:
            Email: "{{ username }}"
            Password: "{{ password }}"            
      - get:
          url: "/message/get-message-count" 

With debug output I"m getting the following:

\   http request: {
  "url": "http://localhost:46551/login",
  "method": "POST",
  "headers": {
    "user-agent": "Artillery (https://artillery.io)"
  },
  "json": {
    "Email": "dev",
    "Password": "xxx"
  }
} +807ms
  http:response {
  http:response   "cache-control": "no-cache",
  http:response   "pragma": "no-cache",
  http:response   "content-type": "application/json; charset=utf-8",
  http:response   "expires": "-1",
  http:response   "server": "Microsoft-IIS/10.0",
  http:response   "unauthenticateduser": "true",
  http:response   "x-aspnet-version": "4.0.30319",
  http:response   "set-cookie": [
  http:response     "Fuze=66DEF75C1CFA4A5691E24A4D37BA7AFF837C5A9F6BC255BAA25DA894080720F6C6C12917FAB059C3D0DB80A204E9FAEB9F95860B30D37C807A66ECAB5747ADD788E3AE01D1F608DCE473933779BA060670B9B1915363EAD7073A94A0C1D0E5A0FDEEA5E45720E3B38EE155E9718C91A6A11084CED1BE5D67ABF00F61E21AB1C10BD1DF71076112F590F767EC57B7E510100C6824FC73C14ED54E60CC70800A6B15A7ACCA; domain=localhost; expires=Tue, 11-Sep-2018 22:36:22 GMT; path=/; HttpOnly",
  http:response     "profile=29; domain=localhost; path=/; HttpOnly"
  http:response   ],
  http:response   "x-sourcefiles": "=?UTF-8?B?QzpcY29kZVx3b3JrXGNhcmVzdGFja1xDYXJlU3RhY2suVW5pZmllZC5tc2hhcGlyb1xDYXJlU3RhY2suQmFja2VuZFxDYXJlU3RhY2suV2ViXGxvZ2lu?=",
  http:response   "x-powered-by": "ASP.NET",
  http:response   "date": "Tue, 11 Sep 2018 22:00:21 GMT",
  http:response   "content-length": "42"
  http:response } +1ms
  http:response {
  http:response   "Version": "v2.7.0",
  http:response   "IsLoginSuccess": true
  http:response } +16ms
  http captures and matches: +2ms
  http {} +1ms
  http {} +2ms
  http request: {
  "url": "http://localhost:46551/message/get-message-count",
  "method": "GET",
  "headers": {
    "user-agent": "Artillery (https://artillery.io)"
  }
} +14ms
  http:response {
  http:response   "cache-control": "no-cache",
  http:response   "pragma": "no-cache",
  http:response   "content-type": "application/json; charset=utf-8",
  http:response   "expires": "-1",
  http:response   "server": "Microsoft-IIS/10.0",
  http:response   "unauthenticateduser": "true",
  http:response   "x-aspnet-version": "4.0.30319",
  http:response   "x-sourcefiles": "=?UTF-8?B?QzpcY29kZVx3b3JrXGNhcmVzdGFja1xDYXJlU3RhY2suVW5pZmllZC5tc2hhcGlyb1xDYXJlU3RhY2suQmFja2VuZFxDYXJlU3RhY2suV2ViXG1lc3NhZ2VcZ2V0LW1lc3NhZ2UtY291bnQ=?=",
  http:response   "x-powered-by": "ASP.NET",
  http:response   "date": "Tue, 11 Sep 2018 22:00:21 GMT",
  http:response   "content-length": "61"
  http:response } +1ms
  http:response "{\"Message\":\"Authorization has been denied for this request.\"}" +2ms
  http captures and matches: +8ms
  http {} +2ms
  http {} +1ms

From this it shows artillery submitting a request to /login and the server response containing a Fuze cookie. However the next request in the same scenario shows no cookies being sent which results in a 401 being returned (which is expected when not logged in).

Reading the documentation shows

Cookies are remembered and re-used by individual virtual users.

This makes me think that any cookie that the server sets should be automatically replayed by Artillery. Am I missing something?

[hassy]

You're right @KallDrexx, cookies should be re-used automatically. We'll need to investigate.

[pschiffe]

I can still see this issue in Artillery: 1.6.0-29

[PirogovClub]

Hi, I have issue near by this. I need artillery NOT to remember cookies. because I want to send request with same cookies, and ignore one that in response, bu artillery add one that is in the response for me And my one. So I'm getting 2 keys with different value in one request.

[vegerot]

I am still having this issue. Where a "Set-Cookie" is not being set

[JustinWinningham]

This issue appears to still be present in 1.6.1, node.js v13.8.0.

[ben-girardet]

Any news on this issue ?

[hassy]

Would love a reproducible test case to help isolate the issue. As it stands it's hard to try to identify what it might be. Artillery's test suite contains tests that check that cookies are set and sent on subsequent requests as expected, so the issue might only show up under certain conditions, or with a particular server/framework. Any more information would be appreciated.

[ben-girardet]

I'm facing this issue with http-only type of cookie (seems also the case of the original poster). The api sends the following cookie:

# copy-pasted from artillery debug logs
http:response     "refreshToken=90d237d9c8a1d6560ad921294e0e97ea; Path=/graphql; Expires=Wed, 24 Mar 2021 21:14:43 GMT; HttpOnly; Secure; SameSite=None"

The cookie is not present in the subsequent artillery request. I mention the http-only type because I'm suspecting that the issue could come from this...

[hassy]

thank you @ben-girardet, the test suite sets HttpOnly but with SameSite=Strict so that's one possible route of investigation!

[Ivan-Cristian]

I am encountering this exact same issue, and it appears that only samesite=strict cookies are used in subsequent calls when using http-only cookies:

  http:response     "Identity.External=; expires=Thu, 01 Jan 1970 00:00:00 GMT; path=/; secure; samesite=none; httponly",
  http:response     ".AspNetCore.Antiforgery.g24zGvIGEcc=CfDJ8NV1-Q10eKZGgzDURlu0fOOtASAKZl0oueHav0F9lgWK1qgGrv0g5b1uHfgwAolAq8TXggoRAnSGu4XVgeuyaKeBD-6CPddyzW2w6eLY7IOkIWKl-VdBKg9ebtywz6gwa8crUv6hsZ5ELDSyd3YqIcQ; path=/; samesite=strict; httponly",
  http:response     ".AspNetCore.Mvc.CookieTempDataProvider=; expires=Thu, 01 Jan 1970 00:00:00 GMT; path=/; samesite=lax; httponly" ]

Yet in the headers of the request following this, it appears only the samesite=strict; httponly cookie is set:

   RequestJar {
     _jar:
      CookieJar {
        enableLooseMode: true,
        store:
         { idx: { my.app.com':
   { '/':
      { '.AspNetCore.Antiforgery.g24zGvIGEcc':
         Cookie=".AspNetCore.Antiforgery.g24zGvIGEcc=CfDJ8NV1-Q10eKZGgzDURlu0fOOtASAKZl0oueHav0F9lgWK1qgGrv0g5b1uHfgwAolAq8TXggoRAnSGu4XVgeuyaKeBD-6CPddyzW2w6eLY7IOkIWKl-VdBKg9ebtywz6gwa8crUv6hsZ5ELDSyd3YqIcQ; Path=/; HttpOnly; samesite=strict; hostOnly=true; aAge=30ms; cAge=34ms" } } } } } }

[ben-girardet]

I'm not an expert about cookies... but this could come from Artillery origin when sending request. I'm testing a node API with artillery and the request coming into node have no origin set. For a proper evaluation of a strict sameSite it should be able to compare the origin of the request (from Artillery) and the host of the API.

What's strange to me is that origin and host are different (because the host is set but not the origin) but it only works with sameSite strict... I don't get that.

But I confirm what @Ivan-Cristian says: I've tried the exact same setup but with SameSite=Strict and the cookie is well replayed with artillery.

[hassy]

Great detective work folks!🕵️‍♂️ This is heaps of information for debugging and is very promising! I 💜 this community!

[hassy]

@ben-girardet @Ivan-Cristian Could you confirm that when you run Artillery tests and SameSite=None cookies aren't being replayed, the server runs on HTTPS? SameSite=None also requires the Secure attribute to be set, so the cookie will only be sent over HTTPS connections.

https://developers.google.com/search/blog/2020/01/get-ready-for-new-samesitenone-secure

[hassy]

@Ivan-Cristian I can reproduce the behaviour with SameSite=None and Strict when the server does not use HTTPS (this would be the expected behaviour). However, I'm unable to reproduce the issue with samesite=lax as in your example. If a cookie is set with SameSite=Lax and no secure attribute, Artillery is replaying it as expected.

[johnoraclestites]

using version 2.0, I have an authentication post request in a scenario that is correctly saving and replaying the returned cookie for all subsequent requests in the flow, but if I move that authentication post to the before section of the overall script, the cookies are not sent. is this expected?

[hassy]

hi @johnoraclestites 👋 that behavior is indeed expected. cookies would be considered virtual user-specific data and as such do not get passed from a before block to individual VUs.

[Matenko]

Was this ever resolved? I have a request that relies on a cookie being set from a previous requests set-cookie header value but haven't been able to get it working. For comparison, modeling the exact same flow through a series of axios requests works fine.

[Maniutek134]

I couldn't find a topic that is more related to the problem I am struggling with so I decided to write a question here as the thread is still open. One of the responses has 'set-cookie' value in its header which is used by default in cookies in the next request in the flow. Seems like Artillery ignores my custom settings from the config.yaml and overrides it with the values from 'set-cookie' header. Is there any way to disable this and have empty cookies before each request within a scenario?