QuiltMC / rfcs

Repository for requests for comments for proposing changes to the Quilt Project.
Other
61 stars 33 forks source link

Revert RFC 81: active user beacon. #84

Closed AlexIIL closed 1 year ago

AlexIIL commented 1 year ago

Rendered View

A significant number of people don't like the active user beacon. Since I'd rather have happy users than "more likely sponsorships" I'd like to remove the beacon entirely.

While I didn't object to the user beacon when it was considered, I now stand against it and would be happier if it was removed. (I personally disabled it just after implementing it, which is a bad sign in general).

TheEpicBlock commented 1 year ago

I am neutral on this and don't really mind the ping that much. If a significant amount of people dislike it then I'd be onboard with this rfc, but as of now I don't see that necessarily being the case

UndefinedBHVR commented 1 year ago

I disagree with the removal. This isn't a malicious metric to maintain, it doesn't log your IP, it doesn't log your HWID, it doesn't do any of that. It's not a privacy violation.

It just sends a request to the server, the server says "okay" then adds to the table, it doesn't log anything beyond that "someone hit this endpoint". This isn't malicious or problematic, the game itself already does this to several other services on its own, another one isn't a problem especially when it's as non-invasive as this. Again: it doesn't identify you in any way, and it can't verify that you haven't sent a request out before or not.

That said, I think it'd be appropriate for those who are genuinely worried to make it an option at install time to permanently disable this metric, so they could avoid the logging entirely if they really wished.

gdude2002 commented 1 year ago

I don't really have a stake in this as a Keyholder, but I figured it might be appropriate for me to add my thoughts and some context:

I think a much better approach to this is to make it easier to opt out, and to make the information surrounding the beacon more accessible.


I want to end this by pointing out that I'm not sitting here trying to convince people who dislike the beacon that the way they feel is invalid - while it's true that a lot of the reaction to this has been over-the-top (in my opinion), countless people have been burned by relentless tracking, put in place by websites, OSes and applications that many of them don't really have any chance to replace. That said, I do feel like it's unfair to project that onto Quilt as a slippery slope argument - there is absolutely no reason at all that Quilt should ever be interested in any kind of invasive tracking, and you can read the source for the beacon project for yourself to note that it quite literally only stores timestamps when it's hit.

The rampant tracking of users across applications and the web, and the extraction of their data, is a significant and real problem. This, however, is anything but that.

LambdAurora commented 1 year ago

Thanks gdude for summing up my own thoughts as an Admin.

OroArmor commented 1 year ago

I would also like to thank gdude for his statement as it also summarizes my thoughts.

duplexsystem commented 1 year ago

Although not strictly the scope of this RFC, I would be much more on board with the active user beacon if it was not proxied through Cloudflare.

Madis0 commented 1 year ago

I don't think this active user beacon is specifically bad, but a lack of an easier opt-out method is bad. Setting an environment variable is an overkill for this purpose and JVM arguments are not obvious or flexible enough, especially when the loader already has a full-featured settings file implemented.

My suggestion is discussed in detail at https://github.com/QuiltMC/quilt-loader/issues/334.

So, I propose either adding a third way to disable this or removing it entirely.

Scrumplex commented 1 year ago

it doesn't log anything beyond that "someone hit this endpoint".

While I am not against the beacon itself, I am not so sure about the factual accuracy of this statement. From what I can tell, the endpoint is either hosted at Cloudflare (Workers?) or is at least protected by Cloudflare. This usually means that information such as source IP addresses are stored for analytics purposes.

Obviously, you can't meaningfully track individual user activity just using their IP address. Especially if the beacon is sent roughly once a month and user's get assigned new dynamic IPs by their ISPs regularly.

The beacon could be more concerning for dedicated servers, as those IP addresses are usually static, which means that usage could be tracked long-term If the presence of this beacon is communicated well to users, I don't see an issue here either, though.

gdude2002 commented 1 year ago

It's a python script, which you would have seen if you clicked the source link I provided - not CF Workers at all!

CF doesn't actually log IP addresses for analytics, as far as I'm aware. When I've worked with CF analytics, it provides anonymized statistics, and I've never seen an IP exposed in that panel.

CF does provide the original IP address in a header as part of the forwarded request when proxying is enabled, but as you can see from the source I linked previously, nothing is done with that.

The only situation I can see that comes anywhere even close is one where rate limiting is applied by IP address, but this only requires short term storage and would be done on CF (eg via a worker) and not on Quilt's infra.

Scrumplex commented 1 year ago

It's a python script, which you would have seen if you clicked the source link I provided - not CF Workers at all!

I have looked at the code. I have never used Cloudflare Workers myself, so it was just a possibility that it is running on there.

CF doesn't actually log IP addresses for analytics, as far as I'm aware

I just added one of my meme domains to Cloudflare to test this. Will report back once I got it working.

Edit: Okay, apparently I can't see many details (with the Free plan). But I can definitely see "unique visitors" which means Cloudflare is definitely doing some basic fingerprinting here.

Cloudflare dashboard showing that it can track unique visitors

Though the dropdown menu only allows me to show the past 30 days.

gdude2002 commented 1 year ago

I've had a chat with someone who knows more about CF than I do, and I've also done a little research myself - here's what I've gleaned:

From everything that I can learn or dig up, CF's privacy-focused approach means that, for the most part, statistics are aggregated and anonymized. I had access to the web analytics dashboard when I was still on the old outreach team or with the MMPA, and I didn't see anything that could have identified anyone in particular - I'm not entirely familiar with what Quilt is making use of under the sponsorship, however, from what I understand, edge-based analytics do let you drill down into what a specific IP has been up to, if you get to it while that data is still available.

This is important to keep track of in some situations (e.g. DDoS/spam/other attacks) - but CF seems to understand the problem with keeping track of this data for very long. If CF wasn't the org responsible for this, Quilt would likely need to be doing something similar to assist them in fending off potential attacks - better to have a privacy-focused org do it than make that another thing that the already low-spoons infrastructure/outreach teams need to manage (and potentially mess up since this isn't in anyone's area of expertise).

However, that's basically the limit of what I can provide - someone on a relevant team would have to be the one to provide context on what Quilt is actually making use of, and what it has access to.

It may also be worth noting that the RFC in question specifically points out a number of restrictions, which clearly have user privacy in mind:

The server hosted at beacon.quiltmc.org MUST:

  1. Be open-source.
  2. Be auditable by any Quilt developer who wish to, within reason.
  3. Not store any data present inside the request.
  4. Not process any data present inside the request beside what is strictly required to establish the connection, following current best practices and technical feasibility.
Scrumplex commented 1 year ago

Looks reasonable. Besides, I really don't mind the beacon anyway. I was just here to point out that IP addresses are potentially stored, even if temporary.

IThundxr commented 1 year ago

Cloudflare does store/show IPs but only for blocked requests etc in the Security -> Events page

Akarys42 commented 1 year ago

Cloudflare does store/show IPs but only for blocked requests etc in the Security -> Events page

That's just wrong, Cloudflare will store IPs for many more things

IThundxr commented 1 year ago

Cloudflare does store/show IPs but only for blocked requests etc in the Security -> Events page

That's just wrong, Cloudflare will store IPs for many more things

Well that's all I've been able to see from Cloudflare with a free account I'm not sure of any other IP collection they do :P

sylv256 commented 1 year ago

I don't think this active user beacon is specifically bad, but a lack of an easier opt-out method is bad. Setting an environment variable is an overkill for this purpose and JVM arguments are not obvious or flexible enough, especially when the loader already has a full-featured settings file implemented.

My suggestion is discussed in detail at QuiltMC/quilt-loader#334.

So, I propose either adding a third way to disable this or removing it entirely.

I'm going to add my two cents to this. I believe every user has the right to handle their data the way they want to not based on the whims of some modpack creator. If you don't like it, you can disable it yourself, but you should not enforce a (mostly illogical) opinion on another but instead give them the explicit choice to disable it.

Ignoring the obvious misinformation--the only data collected is at what time you log in--, it's the Minecraft launcher's responsibility to handle this type of option. It, therefore, makes sense for it to be an environment variable or Java argument.

falkreon commented 1 year ago

I've really only seen a few complaints, many of which were sparked by a thread containing misinformation and threats to block people who had contrary opinions

Look, we've all talked a lot about the beacon, I don't want to re-litigate this because at this point the team has made up its mind, but this is some real Quilt-level BS right here, gdude.

The thread in question

What part about that was misinformation? Is any part of this mischaracterized? I think the thread went above and beyond the call in responsible disclosure and representing contrary points

Threats to block contrary opinions, yes. It was a PSA, it wasn't here for debating the merits of your system because, as we can all see by now, it's exhausting.

gdude2002 commented 1 year ago

The first toot in the thread in question claimed that the beacon was specifically a user tracking payload sent on every launch - it is an empty payload and doesn't track users, and it is not sent on every launch.

I like Una, but I'm honestly baffled at that - she's well known for examining source code and such when she runs into things like this - and as you claim the official communication was linked in the thread (I don't recall this but it's gone now), that's kind of odd, right?

Either way, Una's post is not the point of this thread, and was a single bullet both because of that, and because Una is not the kind of person to get things like this wrong on purpose. The bullet is there to point out that the strong reaction it caused was likely rooted in those two incorrect points, which I think you can agree were quite a sensational claim.

falkreon commented 1 year ago

The first toot in the thread in question claimed that the beacon was specifically a user tracking payload sent on every launch - it is an empty payload and doesn't track users, and it is not sent on every launch.

You and I still disagree whether an empty payload constitutes user tracking. A zero payload contains enough metadata to raise the alarm.

I agree, the "every launch vs every month" is an oversight, but the ommission didn't cause any ill will whatsoever. I think you underestimate the level of intelligence and critical thinking of your userbase by blaming this on una. The chief complainers I've seen (myself excluded) have all been extremely prominent, well-researched, and technically-skilled programmers that I respect.

gdude2002 commented 1 year ago

A zero payload contains enough metadata to raise the alarm.

It has been shown that regardless of this, there is no user tracking. It's a requirement of the spec as per the RFC, and all source code is available.

The baseline data required to make a connection over the Internet may well be a reason, in some people's minds, to investigate. The investigation is done now, is it not? You'd have to claim serious bad faith at this point to imply that Quilt is collecting anything other than what the evidence is showing you, and even then there's no reason anyone would be interested in doing so - who really wants unnecessary additional legal liability?

I agree, the "every launch vs every month" is an oversight, but the ommission didn't cause any ill will whatsoever.

Given you aren't one of the people that's been dealing with the negative feedback, you're not at all in a position to make this assertion. It's also a little worrying that you appear to equate someone taking a toot at face value to a lack of intelligence or critical thinking, in my opinion - but I'm assuming that isn't really your intention here, right?

falkreon commented 1 year ago

The baseline data required to make a connection over the Internet may well be a reason, in some people's minds, to investigate. The investigation is done now, is it not?

Yup, and by now everyone's made up their minds that was going to. But you have to understand that armed with all of the evidence and good reasoning capabilities, intelligent people are still coming in good faith to opposing conclusions about the safety and usefulness of this. This isn't an argument to remove the beacon, this is a reminder that your opponents aren't misled reactionaries.

Given you aren't one of the people that's been dealing with the negative feedback, you're not at all in a position to make this assertion.

Completely fair. I hadn't thought about the fact that Quilt must have received a lot of negative feedback in nonpublic channels or public channels I don't follow, of which there are many. But I find it hard to believe that clarifying that you're "storing a small amount of information and then contacting a beacon with an empty payload" will incense fewer people than "contacting a beacon with an empty payload".

gdude2002 commented 1 year ago

this is a reminder that your opponents aren't misled reactionaries.

I don't see this as adversarial enough to call people opponents, but I guess this is a fair point.

But I find it hard to believe that clarifying that you're "storing a small amount of information and then contacting a beacon with an empty payload" will incense fewer people than "contacting a beacon with an empty payload".

There are some things I think are more relevant here:

Like yeah obviously reiterating the same thing someone already knew, with the same words, won't change much on its own, but this RFC PR seems unlikely to pass even within its own team - so it's especially important to address the everything else

SquidDev commented 1 year ago

Chipping in here, as I'm aware I was involved in some of the discussion on Discord. Apologies for that, as I definitely did not conduct myself as gracefully as I'd have liked.

Thank you to everyone who's working on adding the prompt to the installer and options to various launchers. The installer prompt was a really good suggestion, and definitely alleviates many of my concerns!

I think one thing I'd appreciate clarity on is who has or would need the MAU count. My understanding is that none of the existing sponsorships have required this, and while CurseForge would be interested, it's not been a hard requirement of them developing Quilt support. If there are organisations that Quilt has been approached where it has been more of a sticking point (and it's okay to disclose them!), I'd definitely find that helpful. My gut feeling is that most people are looking for an order-of-magnitude value, which could definitely estimated from the download count, but I honestly don't know!

Some of the reaction has been because of Cloudflare rather than the beacon itself - specifically some people have stated it'd be trivial for us to not take the sponsorship and use other services

In my case, I do think these are orthogonal issues, just happened to be raised at the same time. My feeling here is that it's definitely not trivial (apologies to give that impression, I know that's not the case - my migration took a chunk of time), but work worth doing. Clearly not now - that would leave a bad taste in everyone's mouth - but I do think worth reassessing at a later date.

OroArmor commented 1 year ago

Due to the recent revelation provided here: https://github.com/QuiltMC/quilt-loader/issues/334#issuecomment-1638083180, I would like to change my stance and move forward with the removal of the beacon. Promising no storage of user data, and for that to be false is a horrible mistake and should be corrected.

cocona20xx commented 1 year ago

Considering the accidental storage of user data beyond pure connection statistics, while I wasn't in support of reverting RFC 81 before, it certainly seems best to do it now, considering it was very much the goal to not store any data on the user other than that the beacon was connected to in the first place.

AlexIIL commented 1 year ago

I've removed the RFC 84 file.

AlexIIL commented 1 year ago

Final comment period ends in 4 days on August 18th.

Scrumplex commented 1 year ago

So can we remove https://github.com/PrismLauncher/PrismLauncher/pull/1357 again?

AlexIIL commented 1 year ago

So can we remove https://github.com/PrismLauncher/PrismLauncher/pull/1357 again?

Yes, but I'd recommend keeping the -Dloader.disable_beacon=true argument since it will prevent older versions of quilt-loader from attempting to ping the beacon. (Which isn't really a problem - it just prevents those versions from printing a stacktrace and performing a DNS lookup).

Scrumplex commented 1 year ago

We could add this JVM arg for affected versions in our metadata. That way, we don't need to hardcode this in the launcher itself.