Cookie parameter doesn't work anymore

sitespeedio / sitespeed.io

sitespeed.io is an open-source tool for comprehensive web performance analysis, enabling you to test, monitor, and optimize your website’s speed using real browsers in various environments.

https://www.sitespeed.io/

MIT License

4.75k stars 602 forks source link

Cookie parameter doesn't work anymore #3314

Closed Ludovic33Fr closed 3 years ago

Ludovic33Fr commented 3 years ago

Hello,

It seems there is a regression on the cookie parameter.

how to reproduce your issue

I have upgraded sitespeed to latest v 16.10.0
I switch on fiddler to catch the request
I run a test with the following command : node bin/sitespeed.js https://www.sitespeed.io/ -n 1 --cpu --cookie TEST_PRIVACY=123456 -vvv
I am looking for cookie into HTTP Header : TEST_PRIVACY into fiddler and it fails whereas i can see the cookie parameter into log sitespeedio :

2021-03-16_17h41_22

Please find in attchment the Fiddler session exported Sitespeedio-cookie.zip

This parameter --cookie is required to monitor the third party and simulate the consent cookie (GRPD issue)

Ludovic33Fr commented 3 years ago

soulgalore commented 3 years ago

Hi @Ludovic33Fr I usually use httpbin to verify. They have different URL that just echo back the values. https://httpbin.org/cookies will print all cookies on the screen:

bin/sitespeed.js https://httpbin.org/cookies -n 1 --cpu --cookie TEST_PRIVACY=123456

Can you try if it works for you?

Ludovic33Fr commented 3 years ago

Hi @soulgalore ,

Thank you for your response. I try with httpbin and i think that the issue coming from the execution context.

In deed with your command : node bin/sitespeed.js https://httpbin.org/cookies -n 1 --cpu --cookie TEST-PRIVACY=123456

It works if i call it from Terminal powershell of visual studio code
However : it fails and i reproduce everytime, if i call the command line into a windows prompt

Maybe it is coming from node version : for me v.14.15.4

Ludovic33Fr commented 3 years ago

To add more informations on this issue :

We reproduce the same issue by calling httpbin in linux command line context
We change the version of sitespeed io recently from 13.2.0 to 16.8.0. With this old version it works

soulgalore commented 3 years ago

Ok, can you help me how to reproduce it on Linux? Is it that example you showed or another that do not work?

When I tried the dockerized version it also works for me: docker run --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io https://httpbin.org/cookies -n 1 --cpu --cookie TEST-PRIVACY=123456 -n 1

Setting a cookie for Chrome uses the CDP protocol. this is the magic: https://github.com/sitespeedio/browsertime/blob/main/lib/chrome/chromeDevtoolsProtocol.js#L131-L141

mploquin commented 3 years ago

Hi,

This works for me on Linux, but I also ran into some issues with cookies, especially when I am using --requestheader option. For example, httpbin shows cookies but they are not really in the broswer :

Overall I have encountered several issues with special characters especially (= and :) Ex: TEST_COOKIE=a=b OR TEST_COOKIE=2020-10-12T08:24:23

Moreover we cannot define a custom domain nor a path with the --cookie option. So the solution for me was to use scripts and the Chrome DevTools Protocol.

soulgalore commented 3 years ago

Hi @mploquin yep its a limit today that you cannot set domain/path. If you have time I PR I can rewove it. In CDP it would be pretty straightforward, but for Firefox we still use https://github.com/sitespeedio/browsertime-extension to set cookies, that is little bit harder to test.

Ludovic33Fr commented 3 years ago

Thank you @soulgalore for your advice. It helps me to troubleshoot.

Best,