Telemetry: Gather data re enabled/default/admin themes

[klonos]

This is part of the #285 meta.

I would like us to gather the following data:

• the name of the default theme (to determine of core/contrib/custom/subtheme)
• the name of the admin theme (to determine of core/contrib/custom/subtheme)
• whether "Use the administration theme when editing or creating content" is on/off

Here are a couple of recent use cases where these metrics would be useful that come to mind:

• We are considering to solicit designers to create a new frontend theme and perhaps also a new admin theme (see discussion in Zulip about that). To that end, it would be useful to know how many people are using what themes, and if it is worth spending time (and potentially $$$) on new designs. No point in doing that if the majority is using custom themes and very few sites are using the core-provided themes.
• There are several discussions and known issues with regards to trying to fix problems in Basis w/o breaking backwards compatibility. Well, if only a handful of sites are using Basis, then it might not be the wisest thing that for so long we have been holding back fixing these issues for the minority of cases, and perhaps we should not be allowing these issues to perpetuate for the majority of new sites.

I'm sure that there are other things that called for this over time, but I cannot recall them all now.

Note: if it is a non-core, non-contrib theme, then we should not be collecting nor exposing any specific data - we should be reporting just "custom".

[stpaultim]

I was just thinking that that theme data would be very high on my list of useful information.

I'd like to know how many sites are using Basis or Bartik. I'd also like to know how many sites are sub-theming Basis?

Is it easy to tell both the default theme and any parent themes?

[ghost]

Here's a PR: https://github.com/backdrop/backdrop/pull/4106

[stpaultim]

Is it possible to test this without behind the scenes access to BackdropCMS.org?

[ghost]

@stpaultim You can test the front-end by going to the PR sandbox and installing different themes, then make sure Telemetry is enabled and check the report page.

As for testing the back-end, there's probably a way to do it by setting up your own site with the Project module and then configuring the correct Telemetry URL, but I'm not 100% sure of the exact steps...

[klonos]

I tried to have a look at this, but the PR has now conflicts that will need to be resolved.

I'd like to also get some data on whether any of the core themes are being used as base themes. That would help us decide if/when themes like Bartik (which I am assuming is not used much) should be moved out of core and into contrib for example.

[olafgrabienski]

That would help us decide if/when themes like Bartik (which I am assuming is not used much) should be moved out of core

IIRC, Bartik is a special case. It's mainly in core to make upgrades from Drupal 7 more easy. When people use it this way (I do), Bartik is used often, but it wouldn't show up as enabled for a longer time.

[klonos]

IIRC, Bartik is a special case. It's mainly in core to make upgrades from Drupal 7 more easy.

I'm curious: is the recommended process to switch the D7 site to Bartik before upgrading to Backdrop (in order to have the same theme in the source and destination site)? ...or is it perhaps that many D7 sites use Bartik as their frontend theme or the base theme for their frontend theme?

In any way, I wasn't implying to remove Bartik in the 1.x cycle - just to have enough data when the time comes to start working on 2.x.

[olafgrabienski]

is the recommended process to switch the D7 site to Bartik before upgrading to Backdrop (...)?

Exactly, see step 9 on https://docs.backdropcms.org/documentation/step-2-preparation.

[klonos]

A few of things to consider:

• the purpose of this metric is mostly to provide insight for the usage (or not) of core themes
• we already have usage stats on contrib themes (number of installs/downloads in each theme page on b.org), so there's no need for that
• perhaps it's not such a good idea to be reporting the exact names of custom themes in https://backdropcms.org/project/backdrop/telemetry (for various reasons)
• we DO need to know which is the base theme used by whichever themes are set as default frontend/admin theme though
• if the base theme is either core, or contrib, we should be reporting the theme by name
• if the base theme is neither core, nor contrib, then we should be reporting just "custom"
• we do NOT need to know the entire "lineage" of all themes/subthemes, especially since for contrib themes we can get that information from each respective project on GitHub
• we DO need to know the "top-most" parent theme though

I'll provide a PR for this based on the above considerations shortly.

[klonos]

• we DO need to know the "top-most" parent theme though

I should elaborate on that ^^: Our current telemetry implementation is rather limiting or "mono-dimensional". Consider the Shasetsu and Tatsu themes, both of which are subthemes of the core Basis theme. If we report their full lineage, we'd end up with these entries in the https://backdropcms.org/project/backdrop/telemetry page:

• Basis
• Lateral
• Shasetsu, Basis
• Tatsu, Basis
• AnotherContribTheme, Basis
• YetAnotherContribTheme, Lateral
• CustomTheme1
• CustomTheme2

If we have custom themes that subtheme any of these themes then in addition to the above list we'll have things like these:

• CustomTheme10, Basis
• CustomTheme11, Shasetsu, Basis
• CustomTheme12, Tatsu, Basis
• CustomTheme13, AnotherContribTheme, Basis
• CustomTheme14, YetAnotherContribTheme, Lateral

So the final list would be this:

• Basis
• Lateral
• Shasetsu, Basis
• Tatsu, Basis
• AnotherContribTheme, Basis
• YetAnotherContribTheme, Basis
• CustomTheme1
• CustomTheme2
• CustomTheme10, Basis
• CustomTheme11, Shasetsu, Basis
• CustomTheme12, Tatsu, Basis
• CustomTheme13, AnotherContribTheme, Basis
• CustomTheme14, YetAnotherContribTheme, Lateral
• ...this can get REALLY wild!!!

All the above will be individual entries in the telemetry page, and that is by using a combination of only a handful of themes as examples - imagine how wild it would get with every single contrib and custom theme and subtheme out there! That would skew the metrics to the point where they will be unusable.

What I am proposing instead is this:

• if it is a custom subtheme, then only report the top-most parent theme (if that theme is core or contrib that is).
• if it is a custom stand-alone theme, then only report "custom".

That would result in the following instead:

• Basis
• Lateral
• Shasetsu, Basis
• Tatsu, Basis
• AnotherContribTheme, Basis
• YetAnotherContribTheme, Lateral
• custom theme
• subtheme of Basis
• subtheme of Lateral

I believe that it should be easy to get consensus on the above. However, this is what I am proposing further (for which I understand that there might be some pushback): denote if this is a custom subtheme vs. a contrib subtheme of the top-most parent and leave it at that. That would reduce the list above even further, to this:

• Basis
• Lateral
• contrib subtheme of Basis
• contrib subtheme of Lateral
• custom theme
• custom subtheme of Basis
• custom subtheme of Lateral

I believe that something like the above will provide a shorter and more "manageable" list of themes, which will provide more useful metrics for the following:

• usage of stand-alone core themes
• usage of stand-alone contrib themes (non-subthemes)
• usage of stand-alone custom themes (non-subthemes)
• which core theme is used the most when subtheming
• which contrib theme is used the most when subtheming

By "usage" above, I mean "which is used as an admin or default theme" - NOT metrics on the usage of each theme. As I said, If we need such metrics, we already have the individual project metrics for that:

Thoughts?

[klonos]

... only report the top-most parent theme ...

An alternative to this ^^, which would be simpler logic-wise and also work fine would be to report the immediate parent of a subtheme, and not go any deeper than that. So instead of this: custom theme, Thesis, Basis ...and instead of what I proposed in my earlier comment: custom subtheme of Basis ...we could do this: subtheme of Thesis

We'd still get far more messier than what I proposed earlier, but not as bad as with reporting the full lineage of subthemes for everything.

[indigoxela]

I would like us to gather the following data...

For which purpose?

Remember, there was some discussion about "ethical data collection" initially, when adding Telemetry (2022). One aspect of that is to only collect data we really need for development decisions, not arbitrary data "because, lol, we're curious".

As the times in IT are, there's everywhere this hidden "opt out of data sniffing", and people start to get annoyed by that. Collecting data makes some people wary. And this won't get any better, it seems. So if we collect too much, or add "lol, curious" stuff, they'll simple turn the Telemetry module off.

So I pledge: let's really, really only collect data we need for actual decisions. CKE4 usage is, jQuery1 usage is - to make informed decisions, when it's OK to remove those.

However, which theme people use is not. Backdrop CMS is built to use different themes - from core, contrib, custom, a subtheme of any of those... Which theme people use is not our business.

[avpaderno]

It could make sense to know if there is a core theme that is not used, to remove that theme from core, although it is more probable a theme is replaced by another theme, which uses a "more modern" style.
Otherwise, knowing which themes are most used should not make any difference, for Backdrop core.

[klonos]

For which purpose?

Remember, there was some discussion about "ethical data collection" initially, when adding Telemetry (2022). One aspect of that is to only collect data we really need for development decisions, not arbitrary data "because, lol, we're curious".

@indigoxela no, it's not "just being curious". There are several reasons actually. [edit: the rest of this was moved to the issue summary - thanks for pointing that @kiamlaluno 🙏🏼 ]

In general though, although I understand any honest concerns abound privacy etc., I do not appreciate the assumptions or insinuations that any suggestions for changes, or any requests for telemetry metrics are being made carelessly/capriciously and without consideration or reason behind them. There are reasons, and the intent has always been the improvement of Backdrop as a product - not some whimsical or childish drive. Just to be clear.

However, which theme people use is not.

That is your opinion, and I respect that. I disagree though. I honestly do not see any reason why people would think that collecting metrics about which theme they are using would be "intrusive" and "taking it too much with data collection". You will see in my previous comments, that I have carefully considered and suggested that if it is a non-core, non-contrib theme, then we should not be collecting nor exposing any specific data:

• if the base theme is neither core, nor contrib, then we should be reporting just "custom"

I hope that the above information provides some clarity.

[avpaderno]

@klonos That should be added in the issue summary; at least, it would be clearer there is a reason for gathering information about the used themes. The issue summary should also describe in which way that information is gathered.

[klonos]

That should be added in the issue summary ...

Done 👍🏼

The issue summary should also describe in which way that information is gathered.

I don't think that we have decided on a specific method yet. I'll try to remember to do that once we have a clear path forward.

[avpaderno]

I don't think that we have decided on a specific method yet.

I was referring to a description like if it is a non-core, non-contrib theme, then we should not be collecting nor exposing any specific data - we should be reporting just "custom" which is now added. At least it is clear that names of custom themes are not showed (which in some cases could also reveal which site is using Backdrop).
I did not expect a detailed description.

[klonos]

We should better get #6574 in first.