Unable to set-cookie with ;

ryanbr commented 6 months ago

Prerequisites

[X] This is NOT a YouTube, Facebook, Twitch or a shortener/hosting site report. These sites MUST be reported by clicking their respective links.
[X] I read and understand the policy about what is a valid filter issue.
[X] I verified that this issue is not a duplicate. (Use this button to find out.)
[X] I did not remove any of the default filter lists, or I have verified that the issue was not caused by removing any of the default lists.
[X] I did not enable additional filter lists, or I have verified that the issue still occurs without enabling additional filter lists.
[X] I do not have custom filters/rules, or I have verified that the issue still occurs without custom filters/rules.
[X] I am not using uBlock Origin along with other content blockers.
[X] I have verified that the web browser's built-in content blocker/tracking protection, network wide/DNS blocking, or my VPN is not causing the issue.
[X] I have verified that other extensions are not causing the issue.
[X] If this is about a breakage or detection, I have verified that it is caused by uBlock Origin and isn't a site issue.
[ ] I did not answer truthfully to ALL the above checkboxes.

URL address of the web page

dach-shop24.de

Description

Trying to set-cookie to fix the cookie consent on dach-shop24.de for essential and external media only

Suggested rule:

! essential and external media only
dach-shop24.de##+js(trusted-set-cookie, datenschutz, 'a:4:{s:10:"essenziell";i:1;s:9:"statistik";s:1:"0";s:9:"marketing";s:1:"0";s:6:"medien";s:1:"1";}')

set-cookie stops on essenziell" in ignores the rest of the filter rule.

Other extensions used

N/A

Screenshot(s)

N/A

Configuration

N/A

gwarser commented 6 months ago

dach-shop24.de##+js(trusted-set-cookie, datenschutz, 'a%3A4%3A%7Bs%3A10%3A%22essenziell%22%3Bi%3A1%3Bs%3A9%3A%22statistik%22%3Bs%3A1%3A%220%22%3Bs%3A9%3A%22marketing%22%3Bs%3A1%3A%220%22%3Bs%3A6%3A%22medien%22%3Bs%3A1%3A%221%22%3B%7D')

With reload:

dach-shop24.de##+js(trusted-set-cookie, datenschutz, 'a%3A4%3A%7Bs%3A10%3A%22essenziell%22%3Bi%3A1%3Bs%3A9%3A%22statistik%22%3Bs%3A1%3A%220%22%3Bs%3A9%3A%22marketing%22%3Bs%3A1%3A%220%22%3Bs%3A6%3A%22medien%22%3Bs%3A1%3A%221%22%3B%7D',,, reload, 1)

; is a special character in cookie values. Use raw value, not what the dev tools shows in beautified form.

gorhill commented 6 months ago

Use raw value, not what the dev tools shows in beautified form.

For sake of convenience I think I will add detection of special characters in cookie name/value and encode with encodeURIComponent when detected.

garry-ut99 commented 6 months ago

ryanbr : Trying to set-cookie to fix the cookie consent on dach-shop24.de for essential and external media only

gwarser : ; is a special character in cookie values. Use raw value, not what the dev tools shows in beautified form.

Which is already mentioned in Wiki :

https://github.com/uBlockOrigin/uBlock-issues/wiki/Resources-Library#trusted-set-cookiejs- : The scriptlet does not encode cookie names and values. As a result, if a cookie's name or value includes ;, the scriptlet will not set the cookie since this may cause the cookie to break.

However:

ryanbr : set-cookie stops on essenziell" in ignores the rest of the filter rule.

Seems like a bug / unexpected behavior, because according to Wiki - the trusted-set-cookie scriptlet should not set the cookie and should just decline it as a whole, but instead it sets a partial/broken cookie.

Also related :

https://github.com/uBlockOrigin/uBlock-issues/issues/2777

gorhill commented 6 months ago

With reload

Right, I just tested the changes to automatically encode if needed and a reload was necessary for the cookie notice to go away.

ryanbr commented 6 months ago

Does this fix the option ; in the trusted-set-cookie example above?

gorhill commented 6 months ago

Yes, with the change, your original filter works, no need to encode as pointed out by @gwarser.

JobcenterTycoon commented 6 months ago

white.market##+js(trusted-set-cookie, cookie-consent, '{"required":true,"optional":false,"analytics":false}')

Doesn’t work as expected with uBlock Origin 1.56.1rc5 Firefox. Instead of

{"required":true,"optional":false,"analytics":false}

It set

%7B%22required%22%3Atrue%2C%22optional%22%3Afalse%2C%22analytics%22%3Afalse%7D

and this will make the filter failing. Test site: https://white.market/

gorhill commented 6 months ago

As per RFC 6265 the characters ", should be encoded but apparently browsers don't care. I will prevent those characters from triggering encoding.

garry-ut99 commented 6 months ago

Given you are already editing the trusted-set-cookie doc in Wiki, and you have updated it in the previous issue accordingly to the previous issue: https://github.com/uBlockOrigin/uBlock-issues/issues/3191#issuecomment-2029851635

this part still needs to be updated accordingly to the current issue, as now the scriptlet encodes automatically :

https://github.com/uBlockOrigin/uBlock-issues/wiki/Resources-Library#trusted-set-cookiejs- : The scriptlet does not encode cookie names and values. As a result, if a cookie's name or value includes ;, the scriptlet will not set the cookie since this may cause the cookie to break.

stephenhawk8054 commented 6 months ago

I'm using 1.57.3b0. At this site, there's a newsletter popup when you first open: https://srajagopalan.substack.com/

I tried to set

srajagopalan.substack.com##+js(trusted-set-cookie, intro_popup_last_hidden_at, $currentDate$)

but it returns Wed%2C%2010%20Apr%202024%2007%3A49%3A18%20GMT.

When I use

srajagopalan.substack.com##+js(trusted-set-cookie, intro_popup_last_hidden_at, $currentISODate$)

it returns literal string $currentISODate$ (not the date as expected).

Would it be possible to have a vararg to determine when to encode the value? trusted-set-cookie is being used a lot, I'm afraid there could be more unseen bugs due to this change.

gorhill commented 6 months ago

$currentISODate$ is for set-local-storage-item only. RFC 7231 date is used for $currentDate$ in set-cookie.

I will exclude as a character requiring encoding since the browser does not seem to care about it.

JobcenterTycoon commented 3 months ago

@gorhill

Can’t set

hetzner.com##+js(set-cookie, __Secure-HO_Cookie_Consent_Declined, 1) firefox rejects the cookie.

Only a trusted set works

hetzner.com##+js(trusted-set-cookie, __Secure-HO_Cookie_Consent_Declined, 1)

gorhill commented 3 months ago

Because https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie#attributes:

__Secure- prefix: Cookies with names starting with __Secure- (dash is part of the prefix) must be set with the secure flag from a secure page (HTTPS).

The trusted-set-cookie automatically add the secure flag.

https://github.com/gorhill/uBlock/commit/b4d8750f445cfa06bebd184a6f3cdb4d73148e72: ; Secure will be automatically used when cookie names starts with __Secure- or __Host-.

garry-ut99 commented 3 months ago

when cookie names starts with __Secure- or __Host-.

On one hand, from the link you provided, __Secure- or __Host- prefixes begin with capital letter, which would indicate that they are valid and require ; Secure only when begin with capital letter.

But on the other hand, what I've actually observed, is that both Firefox and Chromium still reject the set-cookie and require ; Secure, if the cookie name prefixes:

begin with lowercase letter: __secure- or __host-,
or are written in UPPERCASE letters: __SECURE- or __HOST- ,
or are written in MiXeD letters: __SeCuRe- or __HoSt-

which would indicate that the both browsers treat and accept prefixes as valid regardless of their letter size.

If that's true, then it seems that regex in the commit should be changed to case insensitive (flag /i) (I didn't test in uBO to see actual results, only looked at the code).

By the way, looking at results in publicwww.com, cases other than __Secure- or __Host- are rare, but still happens ocasionally, for example (there are more examples of course, I just stopped at some point):

UPPERCASE cases: https://publicwww.com/websites/%22__secure-%22/22

508 035 https://www.tersosolutions.com/ Set-Cookie:__SECURE-PHPSESSID=d5nuhothrl
UPPERCASE cases: https://publicwww.com/websites/%22__host-%22/3

136 561 https://accace.com/ \"http\",\"name\":\"__HOST-GAPS\",\"host\":\"ac 145 753 https://www.impactguns.com/ te=None set-cookie: __HOST-SHOP_SESSION_TOKEN=b 152 234 http://discountelectronics.com/ ite=Lax Set-Cookie: __HOST-fbp=fb.1.16981027026 179 512 https://www.cyclopsoptics.com/ te=None set-cookie: __HOST-SHOP_SESSION_TOKEN=1
UPPERCASE cases: https://publicwww.com/websites/%22__secure-%22/22

235 759 https://www.pflege-durch-angehoerige.de/ \"http\",\"name\":\"__HOST-GAPS\",\"host\":\"ac
UPPERCASE cases: https://publicwww.com/websites/%22__host-%22/5

311 274 https://www.vintagetrailersupply.com/ te=None set-cookie:__HOST-SHOP_SESSION_TOKEN=0 315 116 https://soundtraxx.com/ te=None set-cookie: __HOST-SHOP_SESSION_TOKEN=0 318 561 https://www.cigarhut.com.au/ te=None set-cookie: __HOST-SHOP_SESSION_TOKEN=9
lowercase cases: https://publicwww.com/websites/%22__host-%22/2

60 427 https://www.riddles.com/ secure Set-Cookie: __host-riddles_session=eyJp 69 659 https://canny.io/ 19ad-ad7b3352d0d3","__host":"canny.io"},"redux

Of course when encountering such sites, a cookie can still be set with __Secure- or __Host- and it most likely will work just fine, but what if some scripts on the site will check for existence of such cookies and do actions based on it, and their checks won't be case insensitive (they will check for the exact prefix name, for ex. __HOST-)

uBlockOrigin / uBlock-issues