Open leonidborisenko opened 2 years ago
erosman
commented
2 years ago I can pass firstPartyDomain which would work whether privacy.firstparty.isolate is true or false. It would eliminate the error, but I am not sure how the isolation will be affected. e.g.
const cookies = await browser.cookies.getAll({url, storeId, firstPartyDomain: null});Update: Done in v2.45
Feedback needed ....
leonidborisenko
commented
2 years ago I didn't receive new version through updates yet. But I have some thoughts about fix to share.
I was heavily leaning on your understanding of First-party isolation. Honestly, event after reading in various sources about it, I still don't fully understand its' mechanism. Exactly the same as, I believe, you are.
But I think, that passing null as firstPartyDomain totally circumvents this feature. So it should fix the reported bug, but for the (probably unexpected) price of disabling isolation.
I suggest to revert this behavior. Let cookies.getAll fail by default if First-party isolation is enabled, but provide user (and possibly script author) with means to statically and/or dynamically override this behavior.
As First-party isolation is important to extension user, ability to make decision about set of cookies to pass with GM.fetch must be available for extension user.
So there could be several layers, where firstPartyDomain parameter to cookies.getAll() is produced/consumed:
addCookies() function just consumes some cookies.getAll() details (firstPartyDomain amongst them) provided by extension user. If no such details are provided, firstPartDomain (and some other details) are just not set.textarea in global settings) JavaScript module which exports:
url and storeId as parameters and returning an object that will be merged into details passed to cookies.getAll(url, storeId)url, storeId] and values in form of object that will be merged into details passed to cookies.getAll(url, storeId)Or maybe just function would be enough. Difference between object and function is the difference between static and dynamic decision. But function can just return static object (independently of function parameters).
Object returned by user should be sanitized, so that only firstPartyDomain and partitionKey (see below) are merged to details parameter of cookies.getAll().
In this case script author doesn't have any power over isolation and all scripts with GM.fetch will fail for users with First-party isolation enabled and absent setting of that JavaScript module.
I think it's for better even if it pushes considerate chunk of work to extension user.
If this restriction for script authors should be lifted, in my opinion, it should not be done by allowing to pass parameters to cookies.getAll() from userscript (dynamically), because it's not easily observable by userscript user.
Granting to script author ability to conform with First-party isolation along with transparency of this conforming to user and providing extension user to easily observe and change (restrict) script behavior could be achieved by new @grant metadata entry with JSON object as parameter (with [url, storeId] as keys and susbset of cookie.getAll() details object as values).
For example,
// @grant requestCookies {["https://example.com", "firefox-private"]: {firstpartyDomain: "https://example.com", partitionKey: {topLevelSite: 'https://example.com'}}It should be overridible by user in User Metadata.
According to cookies API / Tracking protection:
Firefox includes features to prevent tracking. These features separate cookies so that trackers cannot make an association between websites visited. So, in the preceding example, ad-tracker.com cannot see the cookie created on a-news-site.com when visiting a-shopping-site.com. The first iteration of this protection was first-party isolation which is now being superseded by dynamic partitioning.
Note: First-party isolation and dynamic partitioning will not be active at the same time. If the user or an extension turns on first-party isolation, it takes precedence over dynamic partitioning. However, when private browsing uses dynamic partitioning, normal browsing may not be partitioning cookies. See Status of partitioning in Firefox, for details.
I think, it means, that extension user should be able to control two (and only two) entries in details objects passed to cookies.getAll(): firstPartyDomain and partitionKey.
erosman
commented
2 years ago I didn't receive new version through updates yet.
v2.45 is not released yet.
firstPartyDomain can be set as:
firstPartyDomain: ''
This is the default value when privacy.firstparty.isolate is false but in FireMonkey it is limited to contextual identityfirstPartyDomain: null
Returns all the cookies` but in FireMonkey it is limited to contextual identityfirstPartyDomain: 'example.com'
I need to ask Mozilla engineers about the differenceleonidborisenko
commented
2 years ago In implementing support for first-party isolation, please, consider concrete use-case: enabling "[v] Always use private browsing mode" on about:preferences#privacy page.
In this case contextual identity (storeId parameter of cookies.getAll) will always be represented as firefox-private while containers will not work because of Private browsing.
See:
Containers are disabled in Private Browsing windows and when Never Remember History is selected in your privacy settings.
Even though passing firstPartyDomain: null will still mean "Returns all the cookies' but in FireMonkey it is limited to contextual identity", in reality it will be the same as "Returns all cookies".
I might be wrong. Sorry for distraction then.
If you'll use the method of providing user-defined JavaScript function, then I want to note, that I didn't really fully think about set of parameters of this function that will be useful for extension user to make decision. It might be useful to additionally pass new URL(document.location), metadata block of user script etc.
erosman
commented
2 years ago Please note that AFA Firefox contextual identities, Private browsing is a type of contextual identity (e.g. a special type of container), as mentioned under @container in FireMonkey.
The current situation is:
I am not sure what effect would setting firstPartyDomain have on above (in addition to cookieStoreId).
PS. I changed the setting to '' for now.
const cookies = await browser.cookies.getAll({url, storeId, firstPartyDomain: ''});const url = 'https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch';
const storeId = 'firefox-default';
browser.cookies.getAll({url, storeId}); // Array [ {…} ]
browser.cookies.getAll({url, storeId, firstPartyDomain: null}); // Array [ {…} ]
browser.cookies.getAll({url, storeId, firstPartyDomain: ''}); // Array [ {…} ]
browser.cookies.getAll({url, storeId, firstPartyDomain: 'mozilla.org'}); // Array []const url = 'https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch';
const storeId = 'firefox-private';
browser.cookies.getAll({url, storeId}); // Array []
browser.cookies.getAll({url, storeId, firstPartyDomain: null}); // Array []
browser.cookies.getAll({url, storeId, firstPartyDomain: ''}); // Array []
browser.cookies.getAll({url, storeId, firstPartyDomain: 'mozilla.org'}); // Array []const url = 'https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch';
const storeId = 'firefox-container-1';
browser.cookies.getAll({url, storeId}); // Array []
browser.cookies.getAll({url, storeId, firstPartyDomain: null}); // Array []
browser.cookies.getAll({url, storeId, firstPartyDomain: ''}); // Array []
browser.cookies.getAll({url, storeId, firstPartyDomain: 'mozilla.org'}); // Array []I think, your tests should include case where url is an URL of some third-party which is loaded an set cookies when mozilla.org and some other first-party domain is visited by user. And then firstPartyDomain should be changed in tests to mozilla.org and that other first-party domain, but url should stay the same.
Please note that AFA Firefox contextual identities, Private browsing is a type of contextual identity (e.g. a special type of container), as mentioned under @container in FireMonkey.
I think this feature was proposed in Firefox bug #65965. Quoting from it:
dbaron proposes the following, for separating 3rd party cookies between different 1st party sites:
The idea is this: instead of making local-storage, cookies, etc. stored per-domain of the page setting the cookie/storage (and a domain can only access its own), it could be stored by the pair of (domain of the toplevel page) * (domain of the page doing the setting/getting). In other words, a page on facebook.com inside an iframe when the URL bar displayed cnn.com could only access storage set in the same situation.
It seems like this could help with fixing some of the privacy issues related to third-party cookies and storage while avoiding the risks of breaking sites (other than the need to log in multiple times when you really did want the sharing).
Comment 35 on this bug says:
This bug was resolved by bug 1260931 (implemented the First Party Isolation feature)
FPI could be seen as an automatic "container" (or "sub-container" when actual container is used) with the name computed from hostname (and in some cases also from scheme and port). It contains at least third-party cookies.
Without FPI:
github.com. It loads something from third-party tracking.invalid, which sets cookie.example.com. It also loads something from third-party tracking.invalid. tracking.invalid sees cookie which it set before (at visiting github.com).With FPI:
github.com. It loads something from third-party tracking.invalid, which sets cookie.example.com. It also loads something from third-party tracking.invalid. tracking.invalid doesn't see cookie which it set before (at visiting github.com), because github.com is not example.comcookies.getAllI think, without FPI all cookies, which are set by <some third-party origin> are returned when cookies.getall(<url of that some third-party origin>, storeId) is called.
But with FPI there could be many distinct cookie sets for single <some third-party origin>.
So passing firstPartyDomain computed from <first distinct firsty-party> to cookies.getAll(<url of some third-party origin>, storeId) will get its' distinct cookies 1 and will not get its' distinct cookies 2.
GM.fetchPassing null or '' as firstPartyDomain weakens expected (and explicitly enabled in about:config) isolation.
Well, now I honestly don't know. I might be the only user that is somehow crippled by not supporting FPI and for me {anonymous: true} workaround is just enough. So may be just let cookies.getAll fail (for now) when FPI is enabled and wait for another user to complain, so that use cases will be more clear and support of FPI could be designed more precisely.
When I spoke about computing name of "container", I meant that it's not as simple as taking example.com from http://www.example.com:8081/index.html.
There are much more rules, defined in https://searchfox.org/mozilla-central/source/caps/OriginAttributes.cpp
For example, when privacy.firstparty.isolate.use_site is enabled in about:config, then "name of container" will be equal to (http,example.com,8081).
erosman
commented
2 years ago Maybe not so relevant thoughts about distinctive feature of firefox-private container
It doesn't contradict what I said, although I meant from the browser API point of view of contextual identity e.g. the method cookies.getAll() deals with 'firefox-private'.
- User visit
github.com. It loads something from third-partytracking.invalid, which sets cookie.- Then user visit
example.com. It also loads something from third-partytracking.invalid.tracking.invalidsees cookie which it set before (at visitinggithub.com).
AFA I understand the situation, FPI relates mostly to the browsing behaviour which is not the same for FM HTTP request behaviour. Based on the above scenario:
github.com, cookies will exist for github.com & tracking.invalidexample.com, cookies will exist for example.com & tracking.invalidtarget.com & Contextual Identitiesgithub.com in normal tab, FM lets Firefox handle cookies to target.comgithub.com in private tab, if exists, FM sends cookies created by target.com in private tab to target.com (I don't expect there to be any)github.com in container tab, if exists, FM sends cookies created by target.com in that same container to target.comleonidborisenko
commented
2 years ago Overall, I agree that script author shouldn't be restricted in using HTTP requests basing on browsing behavior.
But in this situation user expectations should be also considered.
FM HTTP request to target.com & Contextual Identities
- On github.com in normal tab, FM lets Firefox handle cookies to target.com
- On github.com in private tab, if exists, FM sends cookies created by target.com in private tab to target.com (I don't expect there to be any)
- On github.com in container tab, if exists, FM sends cookies created by target.com in that same container to target.com
In my current understanding of containers and first-party isolation, theirs' expected outcomes are mostly the same. Both are providing an element to the set of "origin attributes". When browser looks for data by origin, it also uses these origin attributes.
(These examples were edited, because I was using example.com both as first-party domain and origin which is meaningless for first-party isolation)
Without container and first-party isolation origin is: "https://example.com".
For cookies.getAll it's cookies.getAll({url: "https://example.com"}).
In container (and private browsing mode), origin is "modified" by appending container origin attribute: "https://example.com^firefox-private".
For cookies.getAll it's cookies.getAll({url: "https://example.com", storeId: "firefox-private"}).
When first-party isolation is enabled (but container is not used),
origin is "modified" by appending computed first-party origin attribute: "https://example.com^firstparty.invalid".
For cookies.getAll it's cookies.getAll({url: "https://example.com", firstPartyDomain: "firstparty.invalid"}).
When first-party isolation is enabled and container is used,
origin is "modified" by appending computed first-party origin attribute and container origin attribute: "https://example.com^firstparty.invalid^firefox-private".
For cookies.getAll it's cookies.getAll({url: "https://example.com", firstPartyDomain: "firstparty.invalid", storeId: "firefox-private"}).
(Resulting strings of "modified" origin are not the exact representations, but they are close enough to illustrate the described behavior.).
So if FireMonkey limits access to cookies basing on storeId (on container contextual identity), the same logic should be applied to firstPartyDomain.
I want to stress again, that firstPartyDomain value is computed (and not just equals to second-level domain; at least, not in all cases).
Not using firstPartyDomain could break scripts (if user enable first-party isolation), but it doesn't leak cookies in that case. As I'm the only one who report this script breaking and I can live with it, practically not using firstPartyDomain doesn't break anything.
Though using '' or null as firstPartyDomain is an easy fix, it breaks browser user expectation (because it unexpectedly leaks cookies). It's also an exception from FireMonkey respecting "origin attributes" in form of container contextual identities (because passing '' is like storeId: <give me cookies not from this container, but from normal tab> and passing null is like passing storeId: <give me cookies from all containers and normal tab>, if such possibilities existed).
erosman
commented
2 years ago I am not sure the example covers the situation.
TAB that userscript inserted into
https://example.com/example.comUserscript GM HTTP Request from above tab
target.comtarget.com (not example.com)example.com is irrelevant to cookies sent via GM HTTP Request to target.comleonidborisenko
commented
2 years ago Userscript GM HTTP Request from above tab
- URL: target.com
- Cookies sent to above: belonging to target.com (not example.com)
- Cookies sent to above: match the Contextual Identity of the TAB they are sent from
- firstPartyDomain: example.com is irrelevant to cookies sent via GM HTTP Request to target.com
Given that first-party isolation is disabled, firstPartyDomain is irrelevant in cookies.getAll. But it's not the case for which issue was raised. And when first-party isolation is disabled, there is no issue, because then cookies.getAll doesn't fail.
Why storeId is passed in cookies.getAll call?
Given that first-party isolation is enabled by user, why is Contextual Identity is relevant, but firstPartyDomain is irrevlevant?
When user creates new container, he expects that cookies from other containers will not leak into current (new) container. Passing storeId satisfies this expectation.
Maybe storeId is used just for single practical reason: because cookies cannot be passed to requested URL implicitly by browser (as HTTP request is made from extension context and not from container context), so cookies should be retrieved explicitly and so storeId is required, because otherwise cookies will be got from extension cookie jar (from wrong cookie jar).
But still passing storeId provides privacy benefits.
When user enables first-party isolation and loads example.com in new tab, he expects that cookies set by target.com in other tabs with different first-party domains will not not leak into current tab. Passing firstPartyDomain to cookies.getAll satisfies this expectation.
Note that userscript acts on behalf of user, not on behalf of script author.
Script author doesn't care about what cookies are passed to URL requested from userscript. Script author doesn't care about user privacy. User (with enabled first-party isolation) cares about own privacy a lot.
So if firstPartyDomain will not be passed to cookies.getAll, it will fail when first-party isolation is enabled. Userscript will not work and user will be angry. But his privacy will be kept and he can choose between disabling first-party isolation, asking script author to use {anonymous: true} in requests from script or stopping script usage.
If firstPartyDomain will be computed by FireMonkey and passed to cookies.getAll, user expecttions from first-party isolation will be satisfied.
But if '' or null will be passed as firstPartyDomain, user expectations from enabling first-party isolation will be silently broke by FireMonkey and user privacy could be breached.
erosman
commented
2 years ago I have been asking Mozilla engineers for verification. Here is my latest query:
When making an HTTP request to
target.com, from a tab with url ofexmaple.com, what would be relevance offirstPartyDomainto the XHR totarget.com?The first party isolation should relate to
exmaple.com. It should not have any bearing on the cookies sent totarget.com, should it?📌 From a different angle, when making XHR to
target.com& manually getting cookies to matchstoreIdof the generating tab, what should the value forfirstPartyDomainbe set as, so that it covers both the users who have turned itonas well as those who have itoff?ref: Bug 1670278
erosman
commented
2 years ago If you know what to set firstPartyDomain as, let me know and I will update the code accordingly.
browser.cookies.getAll({url, storeId, firstPartyDomain: ''}); Revert it to
browser.cookies.getAll({url, storeId});and leave it be. Otherwise, look at Firefox code for the right answer.
Exact value of firstPartyDomain must be computed at runtime from URL of location loaded in browser tab, where userscript is executed, and according to algorithm, defined in Firefox source code at caps/OriginAttributes.cpp in PopulateTopLevelInfoFromURI function (see it in current version at line 53).
There are two such algorithms (first is a subset of second):
privacy.firstparty.isolate is enabledprivacy.firstparty.isolate and privacy.firstparty.isolate.use_site are both enabled, then after applying first algorithm, the result of it is proccesed furtherFor example, in case when https://example.com is loaded in browser tab:
firstPartyDomain must be example.com if just privacy.irstparty.isolate is enabledfirstPartyDomain must be (https,example.com) if privacy.firstparty.isolate and privacy.firstparty.isolate.use_site are both enabled(But algorithm is not so simple as: just take second-level domain. At least not in all cases.)
If privacy.isolate.firstparty is disabled, firstPartyDomain must not be passed to cookies.getAll at all.
erosman
commented
2 years ago Revert it to
No problem ...
For example, in case when
https://example.comis loaded in browser tab:
That doesn't relate to target.com which is where the cookies are for.
Here is the process:
https://example.com in storeIdGM.fetch('target.com'):browser.cookies.getAll({url: 'target.com', storeId});target.com cookies to target.comWhere does https://example.com relate to getting cookies for target.com?
The only thing the code gets from the tab is the storeId, so that it maintains the same contextual identity.
leonidborisenko
commented
2 years ago Where does
https://example.comrelate to getting cookies fortarget.com?
First-party isolation is a tracking protection. It matters when target.com is an agent collecting data about user behavior. And there should be at least two different first-parties to see how using firstPartyDomain provides protection.
Without first-party isolation:
first-site.com and it makes a request to third-party target.com, target.com can set unique cookie that identifies me precisely.second-site.com and it makes a request to third-party target.com, target.com receives that cookie that is set when i visited first-site.com. It can track me.But with first-party isolation, when I visit second-site.com and it makes request to third-party target.com, browser doesn't send cookies that were stored when I visited first-site.com. There are two isolated cookie jars for first-site.com and second-site.com.
Then. when I'll visit third-site.com, browser will use third isolated cookie jar etc.
The same logic must apply to userscript request made on behalf of user in context of concrete tab. Otherwise it just doesn't meet expectations and guarantees that first-party isolation satisfies and provides.
Speaking bluntly, passing '' or null as firstPartyDomain just breaks first-party isolation and exfiltrate cookies that must be inaccessible.
Let me describe an example in slightly changed way:
1st-site.com, it embeds iframe of target.com, target.com sets unique cookie1st-site.com, it embeds iframe of target.com. target.com doesn't receive cookie that was set when first-party isolation was disabled. target.com sets unique cookie, but it's isolated in separate cookie jar.1st-site.com again, target.com receives that unique cookie from isolated cookie jar, but it doesn't matter, because first-party isolation effect is revealed in case of at least two different first-parties2nd-site.com, it embeds iframe of target.com, target.com doesn't receive that unique cookie that was set when I visited 1st-site.com.3rd-site.com, userscript makes request to target.com, FireMonkey happily sends all cookies from all isolated cookie jars (if firstPartyDomain is null) or cookies that were set when first-party isolation was disabled (if firstPartyDomain is ''). target.com can track me.erosman
commented
2 years ago OK... how can the desired outcome be achieved?
browser.cookies.getAll({url: 'target.com', storeId}); is the same as browser.cookies.getAll({url: 'target.com, storeId, firstPartyDomain: ''}) when first-party isolation is false BUT fails when it is truebrowser.cookies.getAll({url: 'target.com, storeId, firstPartyDomain: 'target.com'}) results in an empty arraySo, what shall be done in GM.fetch to cater for everyone?
leonidborisenko
commented
2 years ago Value of url parameter doesn't have any influence in finding required value of firstPartyDomain.
privacy.firstparty.isolate is enabled or notprivacy.firstparty.isolate is not enabled, don't pass anything as firstPartyDomain (nor even '', neither null).privacy.firstparty.isolate is enabled, determine from extension whether privacy.firstparty.isolate.use_site is enabled or notprivacy.firstparty.isolate.use_site is not enabled, compute firstPartyDomain value by algorithm, defined in Firefox source code at caps/OriginAttributes.cpp in PopulateTopLevelInfoFromURI function (see it in current version at line 53) and pass it as firstPartyDomain. When computing:
aURI is the URL in location bar of tab where userscript is executedaUseSite is false.privacy.firstparty.isolate.use_site is enabled, execute previous step, but in algorithm of computing firstPartydomain select branches assuming aUseSite is true.erosman
commented
2 years ago Parameters in PopulateTopLevelInfoFromURI are not available to extensions.
Since privacy.firstparty.isolate, privacy.firstparty.isolate.use_site are false by default, let's see how they work out when they become true by default and standard feature of Firefox.
leonidborisenko
commented
2 years ago erosman added (feature request) and removed (bug) (done) labels
Can you describe what it means in terms of code? All changes made for "supporting" first-party isolation are reverted and now browser.cookies.getAll (called by FireMonkey background script) again just fails when 'privacy.firstparty.isolate' is enabled, right?
Since
privacy.firstparty.isolate,privacy.firstparty.isolate.use_siteare false by default, let's see how they work out when they become true by default and standard feature of Firefox.
They will not be enabled by default in Firefox ever. However, dynamic first-party isolation (state partitioning + total cookie protection) is already exposed through about:preferences#privacy. It's enabled when "Enhanced tracking protection" is set to "Strict" (see resolved bugzilla bug #1686296).
There is an extensive discussion of what dFPI is in https://github.com/arkenfox/user.js/issues/1051.
Dynamic first-party isolation is related to {partitionKey: ...} and {partitionKey: {topLevelSite: ...}} parameters of browser.cookies.getAll. And topLevelSite should be computed by algorithm defined in the same PopulateTopLevelInfoFromURI (see line 179 of current caps/OriginAttrbutes.cpp).
Parameters in
PopulateTopLevelInfoFromURIare not available to extensions.
Of course, they are not. But their values can be deduced and either ignored (as irrelevant), or got by extension through standard APIs, or requested by extension from user (providing knobs in FireMonkey settings).
aIsTopLevelDocument, aIsFirstPartyEnabled and aForced are only used in computing condition used for decision of early return from function. They should be ignored while early return should be just completely skipped.aOriginAttributes and OriginAttributes::*aTarget is C++-specific method of passing pointer to object member (by passing pointer to class instance and pointer to class member). They are used only once in function in combination for setting variable topLevelInfo which is the actual pointer to object member. Now, topLevelInfo is used as a target for the result of algorithm. So aOriginAttributes and OriginAttributes::*aTarget should be ignored and using topLevelInfo should be interpreted as returning result of PopulateTopLevelInfoFromURI.aURI is the URL in location baraIsFirstPartyEnabled is value of privacy.firstparty.isolateaUseSite is value of privacy.firstparty.isolate.use_siteIt might be productive to reference this issue and ask Mozilla to expose algorithm of computing firstPartyDomain and partitionKey.topLevelSite (defined in caps/OriginAttributes.cpp) through WebExtension APIs.
erosman
commented
2 years ago Can you describe what it means in terms of code? All changes made for "supporting" first-party isolation are reverted and now browser.cookies.getAll (called by FireMonkey background script) again just fails when 'privacy.firstparty.isolate' is enabled, right?
Yes, I reverted the code back to what it was as per #issuecomment-1058365410.
I have already asked Mozilla engineers for the proper way to set firstPartyDomain in cookies.getAll for the purpose of HTTP Request, but was referred to the MDN on First-party isolation.
I am happy to implement the feature, and open to suggestions on how to actually implement it (i.e. the code).
leonidborisenko
commented
2 years ago Yes, I reverted the code back to what it was
Thank you.
I have already asked Mozilla engineers for the proper way to set
firstPartyDomainincookies.getAllfor the purpose of HTTP Request, but was referred to the MDN on First-party isolation.
They pointed in right direction, but in my opinion that information on MDN is insufficient to find how to determine required values in different contexts of multiple sites which URLs will be known only after user browsed to them.
I am happy to implement the feature, and open to suggestions on how to actually implement it (i.e. the code).
I believe I can try to provide you with translation of C++ function to JavaScript function (and supporting JavaScript code) in form of patch to FireMonkey code.
But there is a licensing issue. As I understand, you distribute FireMonkey source code only through addons.mozilla.org and you're fine with people looking at it. But what about modifying it and contributing modifications? Can you put a LICENSE file (with text of license for your source code) in distributed extension extension on next release?
A bigger issue with licensing is that translation of algorithm from Firefox C++ source code to JavaScript code will be a derivative work and it should be attributed so and licensed as Firefox C++ code. Is Firefox code license compatible with your extension code license? The only way to be sure for me is to read LICENSE explicitly distributed by you along with FireMonkey source code.
To put it straight: it's possible that I will not be able to contribute a patch even after including LICENSE, but in absence of LICENSE compatible with Firefox source code, I will certainly not contribute it.
It's not an ultimatum, I'm fine with current situation. Just trying to make this situation clear.
But there could be an easier way (if only taking more time to get through) without any licensing issues.
Try to refer to this issue and ask Mozilla to implement browser.cookies.getOriginAttributes(urlInLocationBar, thirdPartyUrl) function that will use functions from caps/OriginAttributes.cpp and return {firstPartyDomain: ..., {partitionKey: {topLevelInfo: ...}} according to current status of privacy.firstparty.isolate, privacy.firstparty.isolate.use_site, Total cookie protection and privacy.dynamic_firstparty.use_site. When privacy.firstparty.isolate is disabled, there should be no firstPartyDomain in returned object. When Total cookie protection is disabled, there should be no paritionKey in returned object.
In this case you will be able to just merge this returned object to details of browser.cookies.getAll.
erosman
commented
2 years ago But what about modifying it and contributing modifications?
As per AMO page, FireMonkey is distributed under Mozilla Public License 2.0 Open Source Licence.
Getting privacy values require privacy permission and users are really concerned about new permissions (they even complain about harmless ones), especially when the situation only applies to a select group. It that case, often the right approach is to ascertain the popularity & demand.
BTW, have you tested the actual result with privacy.firstparty.isolate true vs false, as per https://github.com/erosman/support/issues/431#issuecomment-1055773108 ?
In my test, there was no difference.
Is there any other userscript manager or other addon that is implementing a similar feature?
erosman
commented
2 years ago I have been thinking about this. TBH, it should not be necessary to use an algorithm to deduce values for in firstPartyDomain in cookies.getAll(). Since it is a setting in cookies.getAll(), then there should also be a standard method to get a value for it, and there should also be guidance on it.
A setting that no one knows what value it should have, is no help.
leonidborisenko
commented
2 years ago I noted your thoughts, but I will answer to your comments sometime later, because for now I want to discuss patch workflow.
I've created a gist (https://gist.github.com/leonidborisenko/5f38c1cd3b38404fa5979dfc05516c37) with work-in-progress patch and instructions on applying this patch. It just contains algorithm translated from C++, but doesn't do anything useful. completed patch (work had been finished at 2022-03-15).
I will update gist in process of further work and post update pings here. Is this workflow suitable for you?
erosman
commented
2 years ago Where is ./vendor/index.js?
Some parts are not necessary for FireMonkey since:
if (scheme === 'about')
only about:blank is possible, not other about:if (scheme === 'moz-extension')
Not applicable to userscriptserosman
commented
2 years ago J have been talking with Mozilla engineers and .........
erosman
firstPartyDomain
When getting cookies in order to pass via an HTTP request to
target.com, from a tab with url ofexmaple.com, what value should be set forfirstPartyDomain?While I understand the function of
firstPartyDomainin browsing (getting/storing cookies), its function in XHR is unclear.The mentioned values of
''ornullboth bypassfirstPartyDomainand therefore defeat its purpose.
Reply
That field is only relevant if first party isolation is enabled, which is a nonstandard and unsupported feature hidden in aboutconfig
erosman
True ... I came across it since some users have it enabled. Tor browser also has it enabled. With the
privacy.firstparty.isolateenabled,browser.cookies.getAllthrow an error which breaks the execution.Therefore, I wanted to know how to set a value for it.
Reply
Ah okay. In your example it should be
example.com
erosman
For my better understanding, how would
example.comfactor in an XHR totarget.com? Only cookies belonging totarget.comare being queried and sent.PS. The whole reason for this process is due to open Bug 1670278
Reply
the cookies would only be included if the firstPartyDomain also matches (that's how it ought to be anyway and roughly what we're doing for TCP)
erosman
How does that work?
Reply
if the Principal is
target.com^firstPartyDomain=example.comandtarget.comsets a cookie, we'd use that Principal to store it and only if the Principal matches on a subsequent request would retrieval of the cookie work
(() => {
const url = 'https://greasyfork.org/en';
browser.cookies.getAll({url})
.then(c => console.log('no-fpd', c))
.catch(console.log);
browser.cookies.getAll({url, firstPartyDomain: ''})
.then(c => console.log('empty-fpd', c))
.catch(console.log);
browser.cookies.getAll({url, firstPartyDomain: null})
.then(c => console.log('null-fpd', c))
.catch(console.log);
browser.cookies.getAll({url, firstPartyDomain: 'google.com'})
.then(c => console.log('domain-fpd', c))
.catch(console.log);
})();// privacy.firstparty.isolate: false
no-fpd Array(3) [ {…}, {…}, {…} ]
empty-fpd Array(3) [ {…}, {…}, {…} ]
null-fpd Array(5) [ {…}, {…}, {…}, {…}, {…} ]
domain-fpd Array []
// privacy.firstparty.isolate: true
Error: First-Party Isolation is enabled, but the required 'firstPartyDomain' attribute was not set.
empty-fpd Array(3) [ {…}, {…}, {…} ]
null-fpd Array(5) [ {…}, {…}, {…}, {…}, {…} ]
domain-fpd Array []Besides the error in 2nd test with no-fpd, the results are the same.
Can you run the test (from FireMonkey Toolbox via about:debugging#/runtime/this-firefox -> Inspect) with different domains and see if the results are different since you have privacy.firstparty.isolate: true all the time?
leonidborisenko
commented
2 years ago While I understand the function of firstPartyDomain in browsing (getting/storing cookies), its function in XHR is unclear.
When Firefox selects cookies for the purpose of sending them in HTTP(S) request initiated by Firefox itself in context of browser tab, it automatically computes and uses firstPartyDomain (when FPI is enabled).
But when HTTP(S) request is made from FireMonkey background page, Firefox doesn't know the exact tab where this request is actually started, so if FireMonkey wants to select cookies with providing first-party isolation guarantees, then firstPartyDomain must be computed manually.
When getting cookies in order to pass via an HTTP request to target.com, from a tab with url of
exmaple.com, what value should be set for firstPartyDomain?Reply Ah okay. In your example it should be
example.comReply if the Principal is
target.com^firstPartyDomain=example.comandtarget.comsets a cookie, we'd use that Principal to store it and only if the Principal matches on a subsequent request would retrieval of the cookie work
Do you understand why firstPartyDomain should be equal to example.com when URL in tab's location bar contains example.com?
For example, do you agree that when https://childcare-support.tax.service.gov.uk/par/app/trialmessage is loaded in tab, then firstPartyDomain parameter of browser.cookies.getAll for getting cookies to send in any HTTP(S) request made in context of this tab should be equal to:
tax.service.gov.uk (when privacy.firstparty.isolate is true)
or to (https,tax.service.gov.uk) (when privacy.firstparty.isolate is true and privacy.firstparty.isolate.use_site is true)?
(Because service.gov.uk is included in public suffix list.)
Do you understand why there are two domains in Principal and how first domain should be determined? Do you understand how Principal value is used by Firefox for selecting cookies from cookie jar to send in HTTP request? How Principal will look when FPI is disabled?
Without FPI Firefox uses only target.com (i.e. requested URL) to select cookies to send in HTTP request. With enabled FPI Firefox computes firstPartyDomain from URL in location bar and stores it along with every received cookie. Then, before HTTP request Firefox computes firstPartyDomain again (from URL in location bar) and sends only cookies that have matching firstPartydomain.
And computing firstPartyDomain doesn't mean "just take second-level domain", even though it looks like it in some cases.
When FPI is disabled, then firstPartyDomain is not required by browser.cookies.getAll at all. So any value of firstPartyDomain (either constant, or variable) passed to browser.cookies.getAll is meaningless for the purpose of providing first-party isolation (because FPI is disabled), even though you can get different results for different values and you can get valid behavior with some passed values. Therefore tests are meaningless in this case.
When FPI is enabled, then real first-party isolation is provided by getting cookies with firstPartyDomain dynamically derived from URL in location bar. So there could not be constant value for firstPartyDomain. But to observe effect of FPI you should test with at least two different first-parties and common third-party and some third-party cookies should be actually set (and stored) before calling browser.cookies.getAll.
And with this, I believe, there is no common ground between us to discuss results of your tests. While I did describe my understanding of FPI mechanism and scenarios when FPI effect can be observable, you didn't. You can find mistakes in my reasoning without me even performing any tests, but I can't find any mistakes in your reasoning, because you didn't provide any reasoning that your tests were founded on.
It's not a fault per se, but if you want discussion, then I just struggle to understand:
What actions did you perform before tests to create test environment? If you did perform any such actions, were these actions required? Why? If you didn't perform any such actions, then did you take into account all consequences of inaction (cookies, set in tests with disabled FPI, can leak into results of tests with enabled FPI and firstPartyDomain: ''; enabled "Total cookie protection" can influence results of tests with disabled FPI; etc.)?
Why did you choose these exact values to pass to browser.cookies.getAll?
What results did you expect to see? Why?
Were results of your tests the same as expected results?
If results differed from expected, then how do you explain differences?
leonidborisenko
commented
2 years ago I've finished patch. There is no need for additional permissions, because I use heuristics to determine enabled method of cookies isolation.
As per AMO page, FireMonkey is distributed under Mozilla Public License 2.0 Open Source Licence.
Thanks, I didn't notice that. Now I see that license is also mentioned in descripton on AMO. It's sufficient for me to provide a patch. But it would be nice to include text file with license in distributed XPI and mention name of license in short leading comment in source files.
I have been thinking about this. TBH, it should not be necessary to use an algorithm to deduce values for in firstPartyDomain in cookies.getAll(). Since it is a setting in cookies.getAll(), then there should also be a standard method to get a value for it, and there should also be guidance on it.
A setting that no one knows what value it should have, is no help.
Well, there is a standard method to get a value for firstPartyDomain: function in Firefox C++ code. But it's not exposed in WebExtension API, so at this moment it must be re-implemented.
Comment by Firefox contributor from ten months ago (https://bugzilla.mozilla.org/show_bug.cgi?id=1669716#c10):
- Parsing firstPartyDomain value from extension [...] Another issue is that it's not obvious to extensions that they have to compute the eTLD+1 of a tab's URL to use as the firstPartyDomain.
Getting privacy values require privacy permission and users are really concerned about new permissions (they even complain about harmless ones), especially when the situation only applies to a select group. It that case, often the right approach is to ascertain the popularity & demand.
I totally agree that requiring additional permissions is a no-go. And even privacy permission will not help with getting values of privacy.firstparty.isolate.use_site, network.cookie.cookieBehavior and network.cookie.cookieBehavior.pbmode.
Some heuristics can be used to select appropriate behavior and escape hatch can be provided in form of entry in FireMonkey preferences where user can explicitly select currently enabled method of cookie isolation.
Is there any other userscript manager or other addon that is implementing a similar feature?
No. Issue for Greasemonkey is linked in top post (greasemonkey/greasemonkey#2985). I've created an issue for Violentmonkey (violentmonkey/violentmonkey#1467). Tampermonkey is closed-source now and when it was open-sourced, there was no first-party isolation in Firefox.
Where is ./vendor/index.js?
It's in added-files.zip included in gist.
Some parts are not necessary for FireMonkey since:
I understand that FireMonkey requires only subset of functionality provided by re-implemented function. But I tried to translate function closer to original implementation, so that any new changes in C++ code could be easily mapped to translated code. I believe, it will help to support this code in future.
erosman
commented
2 years ago Thank you for the explanation.
Therefore, it appears that AFA cookies.getAll is concerned the values of '', null, and tab hostname are not suitable for firstPartyDomain.
Please note that while host or hostname can be extracted from tab URL, getting the TLD is not possible without additional library.
leonidborisenko
commented
2 years ago Please note that while host or hostname can be extracted from tab URL, getting the TLD is not possible without additional library.
My patch doesn't work without additional libraries and publicsuffixlist.js is one of them, being used to get "public suffix"/"eTLD" from tab's URL. All required libraries are included in added-files.zip (contained in gist with patch) and imported by ./vendor/index.js in patch. Licenses of these libraries are compatible with MPLv2.
Baerbeisser
commented
1 year ago privacy.firstparty.isolate aka First-Party Isolation was added during the Tor-Uplift Project as Cross-Origin Identifier Unlinkability and was replaced in Firefox 77 by Dynamic First Party Isolation options in the settings. It was never intended to be used outside the Tor Browser and may leak additional trackable data if used with differing setups.
So, in short: You shouldn't have used it, don't use it anymore and don't complain if stuff breaks if you still do. Please ignore every 2nd privacy-enhancement website still telling you to do so.
leonidborisenko
commented
1 year ago So, in short: You shouldn't have used it, don't use it anymore and don't complain if stuff breaks if you still do. Please ignore every 2nd privacy-enhancement website still telling you to do so.
Thanks for your concerns. In fact, amidst discussion of this issue I did extensively research why FPI was implemented. I've summarized my research in gist (see for instance, some of its' sections: 1, 2, 3) linked in https://github.com/erosman/support/issues/431#issuecomment-1061147637. So I am fully able to make my own decisions, whether to use it or not.
And I've not only complained about FireMonkey error, but also have provided a patch. Again, see aforementioned gist. I did all I could to minimize amount of work required from FireMonkey author to resolve issue.
While reported error doesn't occur when dFPI is used, underlying cause still affects dFPI users. Firefox doesn't automatically apply isolation to HTTP(S) requests made from extensions. It should be done manually by extension author. FireMonkey just blissfully ignores enabled FPI and enabled dFPI. It retrieves and sends cookies from unpartitioned storage when userscript makes request to third-party site.
Quote from Browser Extensions > JavaScript APIs > cookies # Storage Paritioning:
When using dynamic partitioning, Firefox partitions the storage accessible to JavaScript APIs by top-level site while providing appropriate access to unpartitioned storage to enable common use cases. [...] By default, cookies.get(), cookies.getAll(), cookies.set(), and cookies.remove() work with cookies in unpartitioned storage. To work with cookies in partitioned storage in these APIs,
topLevelSiteinpartitionKeymust be set. The exception isgetAllwhere settingpartitionKeywithouttopLevelSitereturns cookies in partitioned and unpartitioned storage.
Quote from Browser Extensions > JavaScript APIs > cookies > cookies.getAll() # Parameters
partitionKey(Optional)An object defining which storage partitions to return cookies from:
- if omitted, returns only cookies from unpartitioned storage.
- if included without
topLevelSite, returns all cookies from partitioned and unpartitioned storage.- if included with
topLevelSitespecified, returns cookies from the specified partition storage.
My patch implements isolation guarantee (as much as it's possible by using official WebExtension API) and resolves underlying cause for FPI and dFPI users. With this change, reported bug is also fixed.
Baerbeisser
commented
1 year ago @leonidborisenko: Sorry, my bad. I always tell people to only post after they read the whole thing... 🤦
Firefox has a "First-party isolation" feature.
privacy.firstparty.isolateis a setting available inabout:config.GM.fetchfails when this feature is enabled. I traced failure tobrowser.cookies.getAll({url, storeId})call inaddCookie()function defined incontent/background.js.According to cookies.getAll() API documentation, it has an optional parameter:
A little bit relevant issue in GreaseMonkey: greasemonkey/greasemonkey #2985
Current workaround (if sending cookies is not required) is to call: