saulpw / visidata

A terminal spreadsheet multitool for discovering and arranging data
http://visidata.org
GNU General Public License v3.0
7.66k stars 274 forks source link

Why is VisiData motd enabled by default? #913

Closed m040601 closed 3 years ago

m040601 commented 3 years ago

Small description Very long time happy user of visidata. Only today, and for the first time in years, I noticed something that made me shake my head and pinch myself.

Everytime I start visidata on the command line, a network request is made to either visadata.org or www.visidata.org

Expected result

When I use a command line application to work on my local computer without any need for network connections, I expect it not to make any network connection.

If, there is such a need for a network connection, for whatever reason, be it:

I expect to understand it, think about it, and decide for my self wether I want it, a network request from my computer to the outside world, to happen or not.

In any case, I also expect this kind of functionality, that is a network request, to be opt in and not opt out.

After searching this intensivly this repo and the official docs, I could not find an answer to this question.

Maybe it is there. Maybe it is to be understood "implicitly" by some other knowldege skil that I lack. So maybe It is my own fault for not finding it

In any case, I would like to make the following request:

Here and now, please explain why this network request is made here and how to deal with it.

Furthermore:

  1. Make it explicit and prominent in the documentation, website, man pages, help etc. why that happens and how can one manage it

  2. Make the all thing opt in and not opt out.

  3. Explain what good use was this anyway for Visidata the project itself after all these years. Was it really usefull ? Were the, possibly, noble goals of implementing this, actualy achieved ? This is an honest technical question. I am really curious about this. What optimization, technical benefict or improvement did the software actually get from getting to know the IP address of it's user and the time of the day it was used.


UPDATE1: I have now found this page, https://www.visidata.org/privacy/

So it is indeed my fault, not having been skilled enough to find what is already there.

I only found it by google search. It is difficult to find just by browsing the site. It is part of the site section "Privacy Policy", but it is not in the documentation/reference/help pages.

Who would have thought, that in the 21st century for using a command line application for browsing sqlite databases, one would also have to:

  1. Remember to think about these things and do the mental work, as it might be something already done, decided and imposed on you. Something that you have to work a little bit to get rid of.. That is, something opt out, and not opt in.
  2. Remember to go online, as the installed offline docs might not be enough
  3. Check the website for any tiny print or tucked away "privacy policy" containing important info, that in not in the manual/help/official doc.

UPDATE2: Checking the stuff I put in my computer for answers

After finding my answers in the online Internet, I went to check if I could have found the same answers, offline.

That is, if the stuff, visidata, I stuffed on my computer to use offline contained these answers.

This is in my opinion a reasonable expectattion. After all visidata also uses a kew kb's of my computer to install for example

I used pipx to install visidata, so I tried this on my computer:

grep --ignore-case --recursive "STUFF" ~/.local/pipx/venvs/visidata/

where "STUFF" i anything available online at, https://www.visidata.org/privacy/. Such as:

but I could not find anything.


UPDATE3 Reasons for opt-in and not opt-out

In case this issue is not simply ignored, deleted and quickly forgotten. Here are som more honest personal reasons for this request. I am 100% many other think just like me. Just in case anyone cares.

...This network request can be turned off by adding options.motd_url=None to your ~/.visidatarc....

In all these years usind visidata I never even needed to customize anything in the rc file. This for me, is usually a sign of good designed, well thought defaults software.

Life is short. I already have many things to manage and think about. I dont want another dotfile to manage, when things could be done opt-in and not opt-out. So for now, I just did the right thing and added the domain visidata.org to my hosts block list.

ajkerrigan commented 3 years ago

There's a productive exchange related to this in #850. Linking to the privacy page from the man page might be reasonable too?

I'm just another user here, but counterpoint: someone has invested countless hours on a tool they've put into the open, which benefits both of us and many others. The "cost" is a once-daily http request that is documented, can be disabled, and fails silently in offline scenarios.

m040601 commented 3 years ago

Linking to the privacy page from the man page might be reasonable too?

Of course ! Why isnt' that done already ? Do we need a rebuild of a CI infrastructure, Github Actions and compiler tools to add one line ?

And why linking ? Why cant you include that content on the man page itself ? At the bottom ? It's not that it's gonna fill up and explode my disk drive.

ajkerrigan:

Thanks for pointing me to the related issue. Your arguments, are very reasonable, and I actually also agree with what you express.

But I dont want to take this discussion further. I've also seen this lately happen in other software projects where I would never had imagined telemetry could be a thing.And I already know how some of them will react. And I already know what my further use of that software will be.

I just want to emphasize and repeat, that I have nothing against "the cost" or "the price" someone has put on something and might want to charge me for. It could even not be open source, and "cost" me 1 dollar a month. Or 100 dollar once. Or a sample of my hair. Or a disclosure of my personal music and food tastes. I may want to pay for it.

It's not whether the "cost" is high or low. It's just, what is the "cost", if there is one. I just want to know what it is, before I "buy" it.

So let that "price" information shine in the sunlight, instead of being a microscopic tag, that one needs a magnifying glass to find it and read it.

saulpw commented 3 years ago

Hi @m040601,

It's great that you feel so passionately about this and are willing to spend time telling it like it is. We need people like you to share their privacy concerns so we can come to a shared understanding of what's acceptable and/or reprehensible.

In fact, I have been quite the privacy ideologue myself. I don't use Facebook or Google or Amazon or Apple, and I don't have a smartphone anymore either (still waiting on my LibreM). This is in large part a rebellion against the dystopian panopticon that the software industry has been promoting over the past 10+ years which makes me deeply uncomfortable.

I remember spending several hours one evening with @anjakefala writing up the privacy policy, making sure it was specifically tailored to someone like yourself who cares about these issues. Did you notice "2019-12-05: no longer use Google Analytics" and "This website does not use cookies"?

I acknowledge that the organization and design of our website in general is not ideal and I welcome anyone who has interest and ideas to help us improve that (please!). But the privacy page in particular is linked in the exact same place as it's linked to on every website, and on every page also, so it's probably one of the easier pages on the site to find.

Speaking of ideals, I've had some other ideals too. In my ideal world, users would opt-in to behavior that benefits everyone. In my ideal world, users would talk in public about why VisiData makes them happy. In my ideal world, users would remain level-headed and curious when inquiring about some feature in software that they've found useful for years.

After 40 years, I can make software as ideal as I have time and energy, but I've slowly been realizing that the real world otherwise falls a bit short of my ideals.

For instance, I derive emotional satisfaction from seeing people use this program I've poured years and thousands of hours of love into. If you really want to know, emotional work is the real cost and price of VisiData (and most OSS, in fact). If you want to "pay" me, start posting glimpses into your VisiData usage in places I (and others) can see them. I'm thrilled whenever someone mentions something cool about @VisiData on Twitter, or when users swing by the #visidata channel on IRC and kibbitz about the ways VisiData makes them happy.

But as much as I've tried to exhort, cajole, or convince users to pay this price, they have many other things to manage and think about, as you said. Only a fraction of users ever say anything, positive or negative, and we save and cherish them all. Most users are like you: they just want to get their work done and go have a nice lunch. They derive utility and happiness from this quirky little terminal tool that for some magical reason has been well-designed and painstakingly executed and works like a charm.

Anyway, it's Saturday evening now, and I had been planning to work on VisiData a bit. (It's a neat little feature we're almost done with, a 'memos' sheet so that you can save values and use them in expressions later). Instead, though, I've spent a decent amount of emotional energy processing a VisiData issue that came in this morning, and then a fair bit of time writing this response. I'm not actually that bitter about this, as these thoughts have been brewing for awhile and it's a great opportunity to say this OAFA (Once And For All). Because you're not the only user who shows up for the first time demanding satisfaction after they've used this free software with delight for years without comment.

And to be quite honest, I actually debated with myself about the motd feature for months, because of my personal distaste of telemetry. Finally I decided that it was necessary to balance the emotional equation, and that adding this little wisp of a feature that simultaneously creates a tiny communications channel to the user (for upgrades and bugfixes and other notices) while enabling a minimum of telemetry, was a reasonable way to go about doing it.

And of course the vast majority of people would absolutely not opt-in to this feature. They wouldn't know about it, and even if they did, as you said yourself, you don't want to have a .visidatarc file. Nobody does. In an ideal world software would be configured by reading your mind. You don't want to have to think about the things you care deeply about, so how could I possibly get you to opt into something I care about?

So to mitigate possible concerns like yours (and mine, if I were a user), I kept the implementation deliberately simple so that someone like yourself could see with a minimum of effort that there is no funny business going on. No code execution or complicated machinery, just a single HTTP request and a display of a random line of text.

To answer your question directly: yes, the feature quite serves its purpose. The daily usage curve inspires me on a regular basis to put one foot in front of the other and update the documentation or fix the test for some corner feature that I don't use, when I would rather be developing some new functionality. We were also able to make the decision to upgrade our dependency to a later version of Python, in order to upgrade the Pandas dependency, in part by seeing the distribution of Python versions among users. You are personally benefitting from our "telemetry" if you are using VisiData 2.0 or later, as many of the improvements in that release were investments in a curve that I could see (and can still see) developing on a daily basis.

So in the free (as in both beer and speech) version of VisiData that I publish, the default for options.motd_url will stay as it is, pointing to a collection of short quotes on visidata.org and providing us with our immensely valuable usage data. You have the legal right to fork it yourself, according to the generous open-source license that I have chosen to release under (GPL3), and maintain a version of VisiData for your fellow privacy purists.

Also please note that as you haven't paid any money, you haven't actually "bought" VisiData; you can think of your current usage as an extended trial period. If you do want to buy some VisiData, for $20/mo you can get access to my vdplus private repo which has a few goodies and will be getting more as time goes on.

Thanks again for your comments.