treebeardtech / nbmake

📝 Pytest plugin for testing notebooks
https://pypi.org/project/nbmake/
Apache License 2.0
179 stars 18 forks source link

Is telemetry really needed? #114

Closed maximlt closed 10 months ago

maximlt commented 10 months ago

Hi @alex-treebeard,

I've been looking for a Pytest plugin to run notebooks and found your project to be an interesting alternative to e.g nbval. Looking at the repo I found out that you recently added some telemetry in https://github.com/treebeardtech/nbmake/pull/113. Even if it seems it's possible to deactivate it, I found that very surprising for an open-source project of this dimension. I usually see bigger projects implementing telemetry, they're usually backed by some start-up/company and need data to report progress to their investors. I don't know if this is the case for nbmake. To me at least, this is definitely a red flag and discards nbmake as a tool I would add to my testing framework.

alex-treebeard commented 10 months ago

Hi @maximlt, this is a question I have asked myself quite a bit.

These are the high-level reasons I recently made this change:

  1. It lets me understand the usage of the project better, this is necessary for prioritising issues and generally helping the community
  2. I found that other projects, even non-commercial ones also do this (similarly transparently)
  3. This project now requires a non-trivial amount of resources from (admittedly a tiny) for-profit company

I'm sorry you feel this is a red flag. IMO it reflects a paradigm shift in (a) how people make product decisions in open source project, and (b) the increasingly commercial aspect of open source.

For context: I am working on more powerful adjacent offerings to nbmake. This is financially risky but I do it because I like to serve developers like you -- telemetry helps bound the risk.

From your perspective, does this make you feel any more comfortable with the decision? Is there are better middle ground I could strike here?

E: Disabled telemetry by default. My main goal with this was to verify that this was actually being used (not just downloaded). I determined it was being invoked ~4K every weekday.

alex-treebeard commented 10 months ago

removed the disabled metrics code in https://github.com/treebeardtech/nbmake/pull/115, closing this, thanks for the feedback