.NET core should not SPY on users by default

[ghost]

@blackdwarf @piotrMSFT I am very disappointed to discover that .NET core comes with a hidden and enabled spy utility that reports on its users. (Lakshanf/issue2066/telemetry dotnet/cli#2145). Apparently, MS has learned nothing from the backclash against Windows 10 spying on users. I suspect many will not want to install .NET core for this reason, which is a shame because .NET core is otherwise cool.

[svick]

@rafaelrpinto

I think Microsoft's position has been a pretty clear NO. For example, the second post in this thread by @richlander says:

we need […] usage data in order to help us find all of the rough edges

The telemetry feature is configurable, so you can turn it on/off at any time. It is also scoped, only applying to tools usage, not the rest of the product. We think that this is a good trade-off and recognize that not everyone will like it.

[rafaelrpinto]

@svick I don't see it that way, on the contrary, I believe that was just the beginning of the discussion. Since the initial post the matter evolved to the current suggestion of an opt in approach which hasn't been formally ruled out so far.

Besides that the issue is still open, which make me believe there is still room for discussion.

[p-m-j]

we need […] usage data in order to help us find all of the rough edges

Thank you for installing dotnet core.
Your usage can give us insight to rough edges where we need to make improvements.
If you are willing please hit Y to enable telemetry you can read more about it at {{url}}.

Otherwise hit any key to continue without telemtry

>

It's not Microsoft want telemetry that's the problem, I'm sure many people would be happy to share, it's the way it's being done.

[richlander]

See my recent post -> What we’ve learned from .NET Core SDK Telemetry. I see it is linked above, too.

I would be happy to entertain more opt-out options along the lines of https://github.com/dotnet/cli/issues/6086. I'm going to focus on that. The current model of telemetry is very useful given the usefulness of the data that it produces (as discussed in the blog post). It is very important for our future development that we continue to have access to the data.

I'm closing this issue. We've heard that feedback and are choosing to continue with the same model. I know that's a disappointment to the folks on the thread (and others). We're focussed on building a great application platform and are choosing to make usage data a very key part of that effort.

Full disclosure: dotnet/cli#7283

[kspeakman]

Thanks for all the fish.

[chrisjsmith]

And thus ends 15 years of .Net and C# for me.

[OpinionatedGeek]

Microsoft has talked a lot about its commitment to 'community' with open-source dotnet, but the truth seems apparent now: the community didn't ask for this change, this Issue shows many in the community don't want this change, and Microsoft say they're going to ignore that and do as they please.

I'm not so much disappointed in the decision as in Microsoft itself. I did think they'd changed when they open-sourced .Net and were honestly going to be community-driven. Apparently not.

[chrisjsmith]

Talk about community and commitment merely appears to be cheap marketing. Did we expect more? Really, the only motivation to open source .Net, move SQL Server to Linux and introduce WSL is to confuse the lines between disparate platforms to leverage sales of a subscription model. I was just hoping to take something away from this which I have invested years of my life in; clearly not. Alas that's a discussion for HN rather than here so I'm going to leave it at that.

[ghost]

And thus ends 15 years of .Net and C# for me.

Maybe it is time to think about fork? I thought about Java but it is not an alternative as well... So we can just remove telemetry and create alternative distribution.. what do you think?

[rafaelrpinto]

I was getting interested on this Microsoft movement to Linux and open source, but now it's clear how things will ever be around here.

@hardhub Between Scala, Node.js or Go there is certainly an alternative for what you need. The only thing .NET does better than all these other platforms is to collect telemetry =]

Cheers,

[ghost]

@rafaelrpinto

Sorry bro but no...

Scala is totally Java dependent.. So why do we need Scala if we cannot accept Java? If can accept Java then Java is very good alternative for .Net.

Node.js is absolutely not alternative... due to JavaScript, performance and absence of multithreading.

Go? Why not Rust then? Rust is C-like and designed to be amazing fast. But I think both have small popularity in community. So they have much less frameworks, community support and so on...

[rafaelrpinto]

@hardhub JavaScript performance and absence of multi-threading?

Good luck with .NET bro. It's the right platform for you.

[dori4n]

@richlander: I'm closing this issue. We've heard that feedback and are choosing to continue with the same model.

I would like to point out, that, without informed, prior consent, collecting any personal information is illegal in most jurisdictions. There is sufficient precedent in the EU data protection directives 1995/46/EC, now superseded by 2016/679/EC, as well as similar legislation in the state of New York (not sure which one at the moment) that collection of any data amounts to collection of personally identifiable information, as the time the data is collected and the information about where on the internet it is sent from can often, though sometimes inaccurately, identify a person, and is often treated as such information. The only way you can comply with the legislation is to ensure every user of your software is prompted and asked for consent (which you cannot reliably do, as computers may be in shared use in a manner that should be obvious to you) or to make telemetry collection an opt-in setting as specified in pulls dotnet/cli#7096 and dotnet/cli#7098 . Yes, I understand this means you will most likely not get any or poor telemetry results. You are going to have to live with that, or stop distributing your software in the aforementioned markets. Blocking telemetry requests on a geoIP basis on your servers does not meet the requirement. Feel free to contact the legal department. You may be liable for damages.

[ibqn]

This telemetry prompts are so badly implemented that they break the package installation on debian/ubuntu cp. dotnet/cli#7420

[chrisjsmith]

This demonstrates my point about testing above as well.

If MSFT don't back down then it's clear the community doesn't matter.

[inoperable]

That there even is a discussion about default opt-out is ridicule- and I'm pretty sure that the way telemetry is handled now (on by default) is illegal within the EU. Like some people already pointed out- this should be off by default, and there should be a simple option (see Homebrew, Yo) to disable it and not dig for information within git issues, and...

How about you telemeter this facts and stop spin-doctoring around angry users comments, and do something and not try to justify something (that your employee might enforce)

If I wouldn't read that small opt-out word after installation on macOS I wouldn't even know that you have and will gather any data at all, I can't say that this is obvious and well communicated fact - or is it? Really, I thought that MS Steve 'Monkey-boy' Ballmer era is gone and 2 words OSS and MS won't auto-generate jokes around the dev's table - but here I am again, feeling like it's 2001.

[inoperable]

Just read the telemetry page on dotnet, following the thought process there (and here to a extent), I hope that every Linux distribution will automatically set DOTNET_CLI_TELEMETRY_OPTOUT=1 environment variable system-wide within /etc

[inoperable]

Btw. your installation script is sending telemetry - before displaying any information about the same step and not as you claim here since your installer launches the dotnet command itself and the very first thing it does, guess what? It phones home with telemetery (and /dev/nulling the thing itself)! For real?

pkginstaller cmd:

dotnet exec $INSTALL_DESTINATION/sdk/2.0.0/dotnet.dll internal-reportinstallsuccess "$1" > /dev/null

launching the same command from term without passing $1

System.InvalidOperationException: Sequence contains no elements
   at System.Linq.Enumerable.Single[TSource](IEnumerable`1 source)
   at Microsoft.DotNet.Cli.InternalReportinstallsuccess.ProcessInputAndSendTelemetry(String[] args, ITelemetry telemetry)
   at Microsoft.DotNet.Cli.InternalReportinstallsuccess.Run(String[] args)
   at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient)
   at Microsoft.DotNet.Cli.Program.Main(String[] args)

Uhmm... ok.

[realityexists]

I hope that every Linux distribution will automatically set DOTNET_CLI_TELEMETRY_OPTOUT=1 environment variable system-wide

@jankun I've raised an issue about this for my Linux distro (Linux Mint): https://bugs.launchpad.net/linuxmint/+bug/1720110 and I hope others do the same

[chrisjsmith]

Good idea.

More deafening silence from MSFT even though the telemetry is flawed as outlined by @jankun

[dazinator]

@chrisjsmith @jankun I know this issue is closed but it's quite an interesting topic so just wanted to leave some feedback. I agree that there should be a checkbox on the installer. The only reason not to have a checkbox on the installer (or any other kind of Yes/no experience), is because it makes it too easy for users to express a desire to opt out. You want to make this difficult, right?. Even if its a small step to go and set an environment variable - 'it's still a step!'. This will mean you still get the data for the portion of people who are too lazy or simply forget to follow up and set that environment variable. That's more data for you.

Also we are trained these days to bypass lots of boring text, and look for the things that we need to engage with on forms. When running through an installer - does anyone actually read the end user licence agreement in it's entirety? Ofcourse not! We have been trained to scan for the UI elements that actually require engagement of some kind. So:

• A bit of text explaining some stuff about telemetry - meh - not important - you can't change this text or have any impact on it right now.
• A checkbox confirming whether I want telemetry enabled? Ok thats actually engaging me with a question. I'll respond.

Microsoft don't want to truly engage you with this question - they need this data. This data makes the product better. That means they don't really want you to have a choice.

It's a shame because when users are given a choice, and they do choose to help improve a product - you garner loyalty from that user at the same time. The user, feels like they chose to help the product and this in turn makes them feel valued - which builds trust, and loyalty. The question may as well ask "Do you trust me?". Microsoft are too scared to ask such a question - they would rather you didn't even think about this question - just focus on the goodies that come in each new release.

Can we blame them? Well we can stop using their product. Thing is dotnet core is absolutely fantastic and this isn't a sacrifice I am willing to make for the sake of what is actually being collected. It's a trade off. I am forced to accept a seemingly underhanded way of collecting data just because the platform is something I want to use.. Their licence is pretty clear on the topic:

License The Microsoft distribution of .NET Core is licensed with the MICROSOFT .NET LIBRARY EULA. This license includes the "DATA" section to enable telemetry (shown below). .NET NuGet packages use the same license but don't enable telemetry (see Scope). DATA. The software may collect information about you and your use of the software, and send that to Microsoft. Microsoft may use this information to improve our products and services. You can learn more about data collection and use in the help documentation and the privacy statement at http://go.microsoft.com/fwlink/?LinkId=528096. Your use of the software operates as your consent to these practices.

Given that the installer records telemetry on install though - it's arguable whether you can "use the software" (and thus consent to telemetry) before it's actually installed. So sending telemetry on install seems a bit shady - depending on whether that classes as actually having used the software yet.. I imagine that "use" the software means as soon as you launch the installer, not as soon as you run your first dotnet cli command. In which case, the installer can send whatever telemetry home it likes, before you are even aware of it, let alone before you decide to disable it.

[chrisjsmith]

It's very shady and the reason it's shady is because no one gives a crap if they get their data, thus proving again that the customer, even if paying like my good self, comes second.

The problem is not that there should be a check box in the installer. It is that the default assumption should benefit the user before the vendor, particularly as this is supposed to be a good citizen open source project. In this case, the telemetry should default to OFF to protect the user's privacy. It should ask to enable it. This applies particularly to the scenarios where it isn't deployed on top of windows and is instead deployed from package repositories on Linux distributions. If you look at the common ecosystems around Debian for example you will not find telemetry enabled by default anywhere. In fact you will see a very strong pro-privacy stance.

Personally, I really don't care if the data benefits Microsoft. It's whether or not the collection benefits me and it does not. There are far better ways to engage the customer and take feedback. Like not letting things sit on MS connect for 5 years unfixed, being able to coordinate between the dotnet and clickonce team on critical issues that crippled us for years, actively listening to customer requests on privacy rather than deafening silence while at the same time trying to drown everyone in blogs, marketing etc. Or perhaps fix some well recorded bugs where the start menu stops responding to the keyboard in Windows 10 that has been floating around for 2 years. None of these things get fixed with data; only customer engagement and not the lip service of blog posts, uservoice and other ridiculous abstractions to try and hide the incompetence.

This is merely a growing abusive relationship with a rather regressive, data-obsessed corporate America and I'm out. There are better options on the table.

If you want an illustration of how much data is being collected, install Fiddler on your PC and let it run for a couple of hours. I personally, and many of us, don't want this to be a de facto standard for software going forwards.

[omajid]

This applies particularly to the scenarios where it isn't deployed on top of windows and is instead deployed from package repositories on Linux distributions.

To be fair, most Linux distributions compile software from source. If they were building .NET Core and packaging it, they could just disable telemetry by default.

[ghost]

@omajid

To be fair, most Linux distributions compile software from source. If they were building .NET Core and packaging it, they could just disable telemetry by default.

I do not agree. Maybe RHEL will do that and maybe CentOS maintainers (in the future) will include own packages in their repositories (with telemetry disabled?). In any case current official documentation from MS says the following about CentOS:

And obviously this package contains original version with telemetry.

There is same situation with Debian.. just look it here for more details: https://www.microsoft.com/net/core#linuxcentos https://www.microsoft.com/net/core#linuxdebian

[omajid]

Maybe RHEL will do that and maybe CentOS maintainers (in the future) will include own packages in their repositories (with telemetry disabled?).

RHEL already does. I am working with CentOS folks to make that happen too (with telemetry disabled by default).

And obviously this package contains original version with telemetry.

Yes, that is true. Sorry, I didn't mean to disagree with anyone here. I was speaking about a hypothetical future where .NET Core was built by and part of Linux distributions. I know that this is not the case right now and the packages published by Microsoft do not meet the packaging guidelines (including collection of user data) expected by many Linux distributions.

But if .NET Core was packaged into Debian (not just for Debian), it would have to comply with packaging standards that Debian uses. And so it probably would have been patched to not collect user information by default.

[realityexists]

RHEL already does. I am working with CentOS folks to make that happen too (with telemetry disabled by default).

That's great to hear! I hope it also happens for Debian and Ubuntu and all their derivatives.

[kstarikov]

RHEL already does

Does it also disable the telemetry during installation that is independent to runs before the message about DOTNET_CLI_TELEMETRY_OPTOUT?

[benaadams]

Does it also disable the telemetry during installation that is independent to DOTNET_CLI_TELEMETRY_OPTOUT?

Telemetry is disabled in every situation, including installation if the DOTNET_CLI_TELEMETRY_OPTOUT flag is set, as in the source for the Telemetry.cs component

[ghost]

I was speaking about a hypothetical future where .NET Core was built by and part of Linux distributions.

It is what I mentioned...
For now any regular user without enough background is affected.
Of course I agree the future can be better than nowadays :)

I am working with CentOS folks to make that happen too (with telemetry disabled by default).

Thanks for that!

In any case I think @kstarikov has asked interesting question.

[chrisjsmith]

@benaadams I'm getting a bit tired of the parroting of MVPs. If you say something over and over and over that doesn't change the point. Seeing as we're verging on the ridiculous evasion of the problem, lets throw a ridiculous analogy at the wall:

This is like walking into a public toilet and being mugged because I didn't write a magic symbol on the wall in one of the cubicles. Look at the problems with this:

1. I'm being mugged before I get the opportunity to write the magic symbol.
2. No one, apart from perhaps in Harry Potter, wants to spent the entire day working out random things to write on the toilet wall.
3. I'm being mugged to start with when every other public toilet is safe to enter.
4. The owner of the public toilets is quite happy because he's keeping all the wallets and phones.

This is a typical example of MSFT's design policy which is fundamentally: stupid defaults and lots of static state.

I've had enough now.

[svick]

@chrisjsmith Someone (@kstarikov) asked a question, @benaadams provided an answer (AFAICT an accurate one). That's it.

Sure, if you fundamentally disagree with the whole thing, discussion about its technical details is probably irrelevant to you. But that doesn't mean others can't have that discussion or that if they do, they are somehow "evading".

[omajid]

That's great to hear! I hope it also happens for Debian and Ubuntu and all their derivatives.

Please get involved and ask others from Debian and Ubuntu to get involved in https://github.com/dotnet/source-build/. This is where the work for building .NET Core from source for packaging is taking place.

RHEL already does

Does it also disable the telemetry during installation that is independent to DOTNET_CLI_TELEMETRY_OPTOUT?

We dont call any dotnet commands in our installation package steps (https://git.centos.org/blob/rpms!rh-dotnet20-dotnet.git/c7-dotnet/SPECS!dotnet.spec). It just copies files around.

[kstarikov]

I guess the issue mentioned here for the OS X postinstall script is that is runs the internal-reportinstallsuccess dotnet command, which sends telemetry before showing the user the message about DOTNET_CLI_TELEMETRY_OPTOUT.

[sushihangover]

FYI: Microsoft

The fact that this is an opt-out vs. opt-in and the opt-out procedure is based upon an env. var. just caused dotnet to fail an externally conducted HIPAA privacy/security audit at a genome testing lab in the backyard of your main campus.

Lucky for us, we were able to convert the runtime to use Mono and get a temporary pass, but this killed the future CIL-based projects with the lab's management...

[chrisjsmith]

@sushihangover this is exactly what I was whining about earlier. There are security policies that this will violate simply for existing.

Now you already screwed windows phone into the ground, let's not do the same with the CLR. It's not like I didn't invest 15 years of my life in the product...

[nathan-alden-inl]

@chrisjsmith I appreciate your passion. I just read your comment on HN and wanted to say I'm equally frustrated about this decisions and others like this. Microsoft, we're talking about command-line tooling for an application stack. There is simply no excuse for this decision. Adopt a policy consistent with how other major vendors like Mozilla expose data collection: opt in only.

As a matter of principle, I've now updated my developer workstation configuration guidelines to require DOTNET_CLI_TELEMETRY_OPTOUT to be set to 1. I've also updated my Cloud Foundry manifests to set it. In addition, I've opened a GitHub issue that asks the Cloud Foundry devs to ensure this environment variable is set for all uses of the .NET Core buildpack.

[voronoipotato]

Microsoft let the train go off the rails again in open source. Their understanding of consent, dialogue, and community really needs to develop or nobody is going to want to work with them. They're going to see Microsoft as the place who can't even cope with free labor.

[jtsagata]

Is there any good fork out there without this crap ?

[sushihangover]

@richlander @terrajobst @migueldeicaza

As a followup to my last post, got the word today that since the dotnet runtime will not allowed to be used in production, the Xamarin.iOS associated project is also dead as any C#/F# code sharing with the backend is now gone.

This was a 25K participant (patients/researchers/doctors) Apple HealthKit/ResearchKit/CareKit Xamarin.iOS app (iPad Pro dashboard for researchers & doctors, iPhone and A-Watch for patients). Killed due to opt-out telemetry that should not exist in the first place, my head is still spinning...

[benaadams]

since the dotnet runtime will not allowed to be used in production,

Build (publish command) a standalone/self-contained app (which the dotnet cli allows you to do) with your dev or build server, then you don't even need the dotnet cli installed on you production machines and there is no telemetry to opt out of

From Self-contained deployments (SCD)

Why deploy a self-contained deployment?

Deploying a Self-contained deployment has two major advantages:

• You have sole control of the version of .NET Core that is deployed with your app. .NET Core can be serviced only by you.
• You can be assured that the target system can run your .NET Core app, since you're providing the version of .NET Core that it will run on.

It also has a number of disadvantages:

• Because .NET Core is included in your deployment package, you must select the target platforms for which you build deployment packages in advance.
• The size of your deployment package is relatively large, since you have to include .NET Core as well as your app and its third-party dependencies.
• Deploying numerous self-contained .NET Core apps to a system can consume significant amounts of disk space, since each app duplicates .NET Core files.

[sushihangover]

@benaadams Thanks for the reply, but a static publish does nothing for this project as the F# genome models are dynamically created/compiled based upon the researchers and doctors queries against the patient's records.

They are taking TBs of graph structured data and morphing it into mobile friendly data that can be visualized "on-the-fly", the genetic researchers have designed/modeled this data and changing that would require a multi-year cycle of government and independent reviews/audits. Even the pre-teen patients get their private data transformed by the same dynamic models into a SpriteKit-based "game" with health goals that are monitored by the parents and doctors.

[raymondSeger]

@ghost You just said what everyone else are thinking.

@chrisjsmith thumbs up

Why would you need to track these kind of data in the first place? Privacy is important to your customers you know!

[ghost]

@sushihangover

Can you explain about Xamarin iOS? Does it have any telemetry as well? Or is it due to integration and code-base sharing?

[sushihangover]

@hardhub In terms of Xamarin, it is due to integration and code-sharing only.

Code-sharing was why a Microsoft stack was considered in the first place, each code module (and resulting assembly in this case) has a massive HIPAA, data accuracy, security, etc.. review and audit process attached to it so "code-sharing" between a dotnet backend and a Xamarin-based mobile solution would have saved about $400k on the project for the first year and another ~$200k per year after that (this post-rollout saving could vary be as much as +/-50% depending upon model/code changes). (This saving is only the review/audit saving, the saving on coding hours was about 2X as over 5000 man-hours was to be saved)

FYI: Xamarin does use telemetry (i.e mono-sgen64 calls out to api.mixpanel.com) but this telemetry is not within any Xamarin.iOS / Xamarin.Android runtime.

[MaikuMori]

I would assume that Microsoft uses .NET Core in their own projects. This should create enough data so that it's not needed to track normal users. At least it would make sense to have opt-in instead of opt-out.

[ghost]

@sushihangover

Xamarin does use telemetry (i.e mono-sgen64 calls out to api.mixpanel.com)

Not sure I understand all exactly... What does "Xamarin" mean here? All Xamarin mobile runtimes do not have telemetry (that you said) and SGen is garbage collector that is implemented for Mono. And you have used Mono to pass audit... hmm.

[sushihangover]

@hardhub Sorry about the confusion (I updated my comment).

mono-sgen64 via Visual Studio calls out to api.mixpanel.com

[chrisjsmith]

Can we stop the language games, the different configurations and madness and uncertainty around which bits of the CLR we are using. We’re not lawyers and don’t want to be.

Just remove it.

[svick]

@sushihangover

a static publish does nothing for this project as the F# genome models are dynamically created/compiled based upon the researchers and doctors queries against the patient's records

Do you actually need the .Net Core CLI for that? If you directly used the F# compiler and possibly MSBuild to compile the models, then I believe there wouldn't be any telemetry.

[emente]

Opt-In or complete removal.

Everything else is unacceptable.