Make sure Element Web works with Firefox ESR

[olberger]

Your use case

It seems (given the discussion in https://github.com/element-hq/element-web/issues/27682, unless the person in charge of support there) that Firefox ESR isn't supported.

Given it has a large user base (enterprise, Debian stable and derived, etc.) it would be very valuable to have it supported.

Have you considered any alternatives?

No response

Additional context

No response

[giplt]

Completely agree. Sure, there are still ways to get to the matrix. But this is rather about the less tech savvy people who don't need bleeding edge features but DO have ethical considerations around using software.

I have been recommending Debian stable as the secure and trustworthy one-stop-go for the most common use cases of communication, office work and web browsing. I also recommended Matrix/Element as the best alternative to proprietary chat applications. And with some success as a fair number of communities I am involved in now use Element for their everyday communication. All of a sudden this combination doesn't work anymore.

Hard as it is to help people navigate software ethics, Element effectively deprecating major open source platforms doesn't help. Most people will never find this github issue and/or tinker their apt repo source lists. I urged people not to judge harshly on the occasional encryption glitches they experience. Just update regularly and things will (and did) improve. But silently withdrawing support and offering a blank web page will make some simply give in to peer pressure and switch (back) to big tech.

[BrenBarn]

This isn't an issue of agility, it is one of policy. If the decision in management is made to support ESR then we will make the necessary changes to support it. Rolling back is not possible as the same update contained a rust crypto update which featured a migration and doing such would corrupt your rust crypto data stores, causing you to need to log out and back in.

That is indeed a problem of policy, but that is still a problem and this issue has drawn attention to it. The general impression I get here is one of backwards decision-making. Why is the policy not to support ESR? Who is making that policy? What is the goal of such policies? It seems to me the way these policies should be made is sort of like this:

1. What audience do we want to reach?
2. What are the characteristics of that audience?
3. What tools do they use?
4. How can we help them, or at least not hurt them?
5. What testing practices are needed to ensure changes don't hurt users?

Instead it seems that platforms were picked for support just because that's what's convenient to develop and/or test on, and then that resulted in not testing on platforms that actual users actually use. But that indicates that the policy-making process is not grounded in something like "what are the things users do, let's be sure we support that".

As was mentioned in other comments, every incident like this does irreparable harm to the reputation not only of Element but of Matrix as a whole. It doesn't really matter what your policies are about what platforms are supported. If users are hurt, they will turn away from your product, more so the more painful the hurt is. Losing access to all of your messages is pretty painful, so something like this has the potential to alienate many users. (Again, not just to alienate them from Element, but to alienate them from Matrix.)

Support policies should be driven by user needs, not the other way around.

[obfusk]

To make a slightly unfair comparison: Android 13 is very much "over a year old" too. And yet I would be very surprised to find the Element Android apps don't work on anything but very recent Android versions. And indeed Element X supports Android 7.0+ and Element supports Android 5.0+. Those are quite a bit older than Firefox ESR.

[wodny]

@oliversalzburg:

In fact, if you only install packages from Debian stable repositories, you're leaving yourself vulnerable to issues that stand unfixed in the stable versions of Debian. Stable in Debian terms does not mean more secure or better in any way, especially if you're on Desktop.

This is false. The stable Debian is always the one that gets timely security updates provided by the Security Team. On the other hand:

Debian Unstable - repository where new & untested packages are introduced. Debian Testing - repository with packages from unstable, if no bug are found within 10 days.

For example compare how OpenSSH CVE-2024-6387 and XZ CVE-2024-3094 were handled and which releases were affected.

[rosemash]

@oliversalzburg @wodny Thanks for linking the evidence. I fear this may be a common misconception and I just want to spell it out even clearer. ESR stands for Extended Support Release because it receives timely updates and security fixes and fixes critical bugs. firefox-esr isn't outdated, it is a mainline branch that is officially supported by both Debian and Mozilla. It gets updates and fixes just as frequently as the main branch does, sometimes even faster. You'd think people in an OSS setting would understand stable release cycles, but if you go back through the thread, it's clear there are a lot of people who just assume the best practice for Debian is to meddle with apt sources and update some packages to newer versions when appropriate. People genuinely don't take Debian seriously as a platform.

[obfusk]

ESR stands for Extended Support Release because it receives timely updates and security fixes and fixes critical bugs. firefox-esr isn't outdated, it is a mainline branch that is officially supported by both Debian and Mozilla. It gets updates and fixes just as frequently as the main branch does, sometimes even faster.

Exactly. If you look at https://www.mozilla.org/en-US/security/advisories/ you can see both regular and ESR Firefox getting fixes at the same time. It just doesn't get new features as often. But that doesn't make it "old".

[jonorthwash]

Wow. I just want to add that this is one of the weirdest FOSS issues I've seen in several decades of using and developing FOSS. The devs breaking support for ESR software, adamantly refusing to find a work-around, and not even writing a little error handling so that users receive a message about it being an incompatible version?

The last part is the weirdest part of this. The in-browser Element client has detection for lack of javascript support in the browser, but doesn't tell you "sorry we don't support Firefox ESR". Instead one day your client says there's a new version available, you let it reload, and you just get a blank page that no amount of clearing cache or using private browser windows will make change.

I expect an occasional glitch or something when using Firefox ESR, like I get in corporate products (looking Google's direction—and they occasionally even have messages saying that they don't really support Firefox), and I expect FOSS devs to be overworked and not always support a particular configuration, but a blank page with no further explanation for all Debian stable and Tor users? It's the worst of both worlds.

[jonorthwash]

Wow. I just want to add that this is one of the weirdest FOSS issues I've seen in several decades of using and developing FOSS. The devs breaking support for ESR software, adamantly refusing to find a work-around, and not even writing a little error handling so that users receive a message about it being an incompatible version?

I believe some of my surprise here is exactly because of what @rosemash stated. Element devs clearly have no idea what ESR means, and treat it simply as "old". There's a disconnect between the development team and the real world that is causing them to fail to understand why this issue is such a big deal. Their attitude has been dismissive from the start.

Devs: take pride in your work, and don't dismiss your users when they tell you that you screwed up.

[t3chguy]

and not even writing a little error handling so that users receive a message about it being an incompatible version?

That has been done, it was buggy in the latest release but the fix will ship in the upcoming release.

Element devs clearly have no idea what ESR means, and treat it simply as "old".

Element devs follow the policy set out by people that manage them, they get an input but they do not control the policy.

[jonorthwash]

and not even writing a little error handling so that users receive a message about it being an incompatible version?

That has been done, it was buggy in the latest release but the fix will ship in the upcoming release.

Why isn't this a patch given more priority? How many duplicate bug reports have you received already because this is still just failing silently? And those are just the users who know enough, have a GitHub account, and care. The rest are just getting confused and assuming the problem is either their "computer" or your product.

[jonorthwash]

Element devs clearly have no idea what ESR means, and treat it simply as "old".

Element devs follow the policy set out by people that manage them, they get an input but they do not control the policy.

How do we get a message to these people that manage the devs? Surely they need a policy that's in line with ESR browsers and that avoids alienating users by giving them a blank page.

[t3chguy]

Why isn't this a patch given more priority?

I made the patch the same day of the first report

[jonorthwash]

Why isn't this a patch given more priority?

I made the patch the same day of the first report

A patch given more priority would've been rolled out then too. So while this information is helpful, it doesn't address the question you appear to be responding to, which I can now make more specific: why has the patch still not been deployed yet? (And this question is kind of less important than the others anyway.)

[t3chguy]

why has the patch still not been deployed yet?

It has been deployed to develop.element.io and staging.element.io. It will be deployed to app.element.io when the release happens later today. 1.11.70 ran late due to issues with rust crypto and this meant there was no gap between 1.11.70 and the following RC for any hotfix releases, our tooling currently only supports maintaining one active release cycle/branch at a time and as the policy stands the issue only affecting (currently) unsupported environments it wasn't a drop everything and make the release process happen sort of thing, as that'd have knock on effects on customer projects and everything else.

[gpcf]

Wow. I just want to add that this is one of the weirdest FOSS issues I've seen in several decades of using and developing FOSS. The devs breaking support for ESR software, adamantly refusing to find a work-around, and not even writing a little error handling so that users receive a message about it being an incompatible version?

Honestly, I'm completely baffled at this attitude, as Element has marketed itself quite a bit as a messenger for institutions. I work in IT at a large German university, and basically the entire department runs on Debian, while the University has set up element as a chat service, and I think I am very much not alone with this kind of setup. This is an issue that affects thousands of users, many of them too non-technical to use the "fixes" suggested by the devs, such as students using a university computer pool. The way this was handled is how you lose users to other platforms, and lose them for good.

[galitus]

I second the post above and would like to add, myself an admin for a bunch of PCs/Laptops, I use the ESR for deployment. Why? Because I don't want to deploy every second week a new release. It is 30 releases from 115.1/116 to 128 for the mainline version and only 16 for the ESR one. And what features do normal people need from the mainline version anyway, to exclude the ESR here?

I like the product matrix/element, and it hurts to see that this kind of development hurts the acceptance and usage in the long run.

[jonorthwash]

I now get the "Unsupported browser" message. Which just tells me to "Please install Chrome, Firefox, or Safari for the best experience." While running in Firefox. Stating "latest version" might help avoid even more confusion.

[t3chguy]

@jonorthwash it is on Design's todo list to redesign that view with better messaging, my fix merely made the existing one work again.

[aabittner]

my 2ct: wonder about this whole situation and people in charge of this project seem? or act as if there didnt know about firefox and their channels and how the whole software industry works out there in the real world, at least thats my taking on this situation. firefox esr channel has been there for years, and obviously the real world doesnt update a piece of software the moment something is released to the public, and there are overlapping time spans of software versions in the real world as well. when this whole matter started of suddenly deprecating the then-current firefox esr 115.x it was absolutely cler that there was no released esr beyond that version range, and esr 128 only appeared a day or two later into this bug, then people in charge in here pointing out merely that firefox (esr) did in fact appear as 128 that very day. anyhow, in reality web developers should know about big browser vendors and about projects making use of these browsers and how they ship their stuff and that they dont abandon an esr channel for example that is absolutely designed and laid out in the way to be overlapping at the beginning/end of the releases obviously.

firefox 115 esr will coexist for several more months now in probably quite a number of linux distros and other situations and surroundings out there (academia, huge institutions etc were some exmples here in this bug ticket) and this is still not a good solution on my opinion when people just cant switch over to esr 128 for a long time just yet.

please consider this matter and this bug for the next installment of ESR of firefox in the future. it is all laid out and known in advance as of today. if plans dont change fundamentally, this will be the same situation again at the next ESR switching to a higher major release level.

thanks.

[konomikitten]

Just an FYI but clicking the Go to element.io link does not work on the new "Unsupported browser" page.

Edit: Also it's saying "Please install Chrome, Firefox, or Safari for the best experience." Which is confusing because I have Firefox installed.

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Firefox/115.0

[MatMaul]

Any chance the changes get reverted for a couple of months only ?

The new ESR is out, but the old one will still be around for some time, until admins upgrade it.

We are heavily impacted on Tchap federation too, and we will have to revert this changes on our side, quite a lot of state deployments use ESR.

[Seirdy]

The Tor Browser has historically lagged on ESR updates due to the amount of patching required, so if anonymous users are an important demographic it's generally a good idea to support an ESR version for a few months after it's EoL. It sucks to have to support EoL software but unfortunately there are no other browsers that provide comparable fingerprinting protections.

[theres-waldo]

Any chance the changes get reverted for a couple of months only ?

Just to make this request more explicit: I know that it was explained in a previous comment that a wholesale revert to a previous Element version is not feasible.

But has reverting just the specific change that introduced the ESR incompatibility (which, if I understood the disucssion correctly, was the introduction of a dependency on Intl.Segmenter in https://github.com/matrix-org/matrix-react-sdk/pull/12617) been considered?

It seems to me that such an action could be taken, to address the current pains in the short term, independently of any larger discussion / commitment about ESR support going forward.

[t3chguy]

But has reverting just the specific change that introduced the ESR incompatibility (which, if I understood the disucssion correctly, was the introduction of a dependency on Intl.Segmenter in https://github.com/matrix-org/matrix-react-sdk/pull/12617) been considered?

That would not fix your issue, as some dependencies, e.g. https://github.com/element-hq/compound-web also use Intl.Segmenter so the same exception would be thrown. The teams responsible for those dependencies would also need to make similar changes.

It seems to me that such an action could be taken, to address the current pains in the short term, independently of any larger discussion / commitment about ESR support going forward.

Yes an action could be taken, if we were tasked with such, at the end of the day our employer chooses what we work on.

[erebion]

In fact, if you only install packages from Debian stable repositories, you're leaving yourself vulnerable to issues that stand unfixed in the stable versions of Debian.

The Debian maintainers do a lot of hard work to fix security issues. Please do not spread FUD.

Aside from that, ESR is not only available for Debian.

Many people have given a lot of excellent examples in response, so I will not add any more.

But has reverting just the specific change that introduced the ESR incompatibility (which, if I understood the disucssion correctly, was the introduction of a dependency on Intl.Segmenter in matrix-org/matrix-react-sdk#12617) been considered?

It seems to me that such an action could be taken, to address the current pains in the short term, independently of any larger discussion / commitment about ESR support going forward.

Yup. It could then be re-introduced once ESR has moved to a newer version with that included.

That would not fix your issue, as some dependencies, e.g. https://github.com/element-hq/compound-web also use Intl.Segmenter so the same exception would be thrown. The teams responsible for those dependencies would also need to make similar changes.

Seems sensible. Do not lock out a large amount of users and for that just delay that a bit until ESR has caught up.

[o5k]

[...] if you only install packages from Debian stable repositories, you're leaving yourself vulnerable [...] [...] your desire to use an ancient ESR [...]

ESR is updated. It is by objective definition not ancient. That's its whole thing. Don't play the already jaded "old = bad" card when the software in question isn't even old. It's getting very tiring to hear this rhetoric of "if you're not running the latest nightly builds you are a luddite".

[o5k]

my 2ct: wonder about this whole situation and people in charge of this project seem? or act as if there didnt know about firefox

@aabittner yeah, that tends to be how it works — a dev pushes an update that breaks compatibility first, then they yap and complain about how everyone having issues is a luddite who needs to get with the times and use the Latest and Greatest you never get week-long defensive back and forths from developers who actively knew what they were doing when pushing a breaking change

[srett]

*sigh* Matrix being Matrix again. I wish we'd use IRC at work, but I guess without emoji reactions you can't be productive in this day and age...

[timmc]

Besides the concerns others have raised here around minimum support and communication, I would implore the devs to take more care to highlight breaking changes in the changelog. I review the changelog every time I go to upgrade my shared Element installation and I rely on it for such warnings.

[erebion]

What was the last compatible version? I struggle finding out where exactly it broke.

[timmc]

1.11.69 is the last one that worked for me. I upgraded our installation 1.11.71 and was confronted with a weird error message from the browser about security headers or something, then tried 1.11.70 and got a different error. Downgrading to 1.11.69 worked, presumably because I didn't get far enough in to have my session corrupted.

[BrenBarn]

Besides the concerns others have raised here around minimum support and communication, I would implore the devs to take more care to highlight breaking changes in the changelog.

I'm not a dev, but my interpretation of their comments here is that this isn't considered a breaking change due to the platform support policy. There is no commitment to keeping the app working on any particular browser version that it used to work on. It's just "the last two versions of major browsers". If new browser versions come out, the old ones aren't supported anymore, so there's not even any attempt to test on those versions, and changes that break them aren't considered breaking.

Personally I deplore this kind of development philosophy and (like I mentioned in an earlier comment) I think it speaks to misguided decision-making at a higher level (see also repeated comments from the devs saying "this was a management decision").

[timmc]

I agree that that might be the philosophy, but I've seen any number of projects note breaking changes for "unsupported" cases.

[knarrff]

Independent of the current mess: can we please get ESRs into the list of supported environments for Element?

Also, just because there is a new ESR out does not mean the old is not supported anymore. There has to be a grace period for distributions and users alike, e.g. while 128 was released 2024-07-09, the EOL of the last EST before that (115) is 2024-10-01. 115 should be supported at the very least until that date by element.

[srett]

Fixed with userscript

// ==UserScript==
// @name Fix the Matrix
// @match https://<your homeserver>/*
// @run-at document-start
// ==/UserScript==

Intl.Segmenter ??= class { *segment(string) { for (const segment of string) yield {segment}; } };

Works in ViolentMonkey but not GreaseMonkey because webdev.

[andrewshadura]

Adapted for Greasemonkey (although not sure why this is needed):

// ==UserScript==
// @name Fix the Matrix
// @match https://app.element.io/*
// @run-at document-start
// ==/UserScript==

let script = document.createElement('script')
script.type = "text/javascript"
script.async = false
script.src = "data:,Intl.Segmenter ??= class { *segment(string) { for (const segment of string) yield {segment}; } };"
document.head.prepend(script)

[obfusk]

Intl.Segmenter ??= class { *segment(string) { for (const segment of string) yield {segment}; } };

Whilst it's a clever trick to unbreak Element not even loading, this isn't an actual replacement for what Intl.Segmenter does. It will produce incorrect results for anything but basic English. It will not handle emoji or any kind of character represented using multiple code points.

It might be a viable workaround for some people -- if you really don't need correct results for all those other cases like emoji -- but it is not a "fix". I wish it was that easy. I thought of doing something similar but even if I tried to take variation selectors and zero width joiners into account there would be far too many cases where it produces the wrong result, even if it would likely work for me.

Which is of course exactly why Element uses Intl.Segmenter. It's absolutely the right choice. Or would be if it was supported in Firefox ESR.

[andreymal]

a viable workaround for some people

This is called “graceful degradation”, a now-forgotten technique of ancient web developers.

[obfusk]

This is called “graceful degradation”, a now-forgotten technique of ancient web developers.

Hopefully, yes. But I don't know what the consequences of the workaround producing wrong results are here. If it's just going to result in rendering some text badly or could actually cause nasty bugs.

[srett]

Whilst it's a clever trick to unbreak Element not even loading, this isn't an actual replacement for what Intl.Segmenter does. It will produce incorrect results for anything but basic English. It will not handle emoji or any kind of character represented using multiple code points.

No, this is a complete reimplementation that I thoroughly validated with millions of unit tests.

It might be a viable workaround for some people -- if you really don't need correct results for all those other cases like emoji -- but it is not a "fix". I wish it was that easy. I thought of doing something similar but even if I tried to take variation selectors and zero width joiners into account there would be far too many cases where it produces the wrong result, even if it would likely work for me.

Which is of course exactly why Element uses Intl.Segmenter. It's absolutely the right choice. Or would be if it was supported in Firefox ESR.

It did just fine without Intl.Segmenter until two weeks ago. The only place where I noticed it being used is those fallback avatars that contain the initial letter of the user. So yes if you don't have an avatar set and your nick starts with an elephant, this might not display correctly. I guess between potentially displaying garbled avatars and not supporting Firefox ESR, the latter is probably the right choice.

But most likely nobody is even really aware what else this might be used for, since according to that comment above, it's not to be found in element's source code directly but, as usual with modern web apps, in a dependency of a dependency of a dependency that's a dependency of left-pad.

I can't rule out this somehow manages to kill your dog or something. But at least when you use that hack, Element still displays a warning that your browser might not properly be supported and you need to acknowledge this by clicking a button. I guess that's something and would make it acceptable to officially include this (at least as a hotfix release, in case someone want's to properly tackle this). As said, the disclaimer is already there. But picking a pragmatic solution isn't cool these days, I know. "Users won't understand that warning, dismiss it and then complain about garbled avatars!"

[giplt]

Could you perhaps add a link to https://matrix.org/ecosystem/clients/ in the error message that is displayed? Until Element gets fixed users can navigate to an alternative web client that will work in their browser without having to upgrade their systems...

[obfusk]

It did just fine without Intl.Segmenter until two weeks ago.

Yes. Because it was using a library to do the same thing. One that produces correct results because it actually splits on graphemes, not code points like your code.

I guess between potentially displaying garbled avatars and not supporting Firefox ESR, the latter is probably the right choice.

I agree that supporting Firefox ESR is worth the risk of potentially displaying garbled avatars. 100%. And I think it's great you've provided that userscript as a workaround.

But most likely nobody is even really aware what else this might be used for.

And that was my point. That we don't know for certain what might break with a replacement that doesn't actually do what Intl.Segmenter does. I'm not saying "don't use it". I'm just saying tell people that the most likely breakage will be something harmless like garbled avatars but that no one knows for sure.

[rosemash]

Yes. Because it was using a library to do the same thing. One that produces correct results because it actually splits on graphemes, not code points like your code.

I don't see any reason element couldn't just continue to use the library it was using before. It seems like there was no reason to hastily update to a bleeding edge browser implementation of something that was previously done by a library. Furthermore, there's no reason it can't return to using the library to continue to support Debian.

[intrigus-lgtm]

I can also offer this script which uses https://github.com/cometkim/unicode-segmenter?tab=readme-ov-file#export-unicode-segmenterintl-polyfill to polyfill Intl.Segmenter.

(If you use uBlock Origin and Tampermonkey, you have to disable uBlock Origin on your home server's url)

[andreymal]

... avoid bloating the bundle with a mass of polyfills and points of potential supply chain attack

Great, now users put random, potentially insecure scripts from the internet instead

[knarrff]

Great, now users put random, potentially insecure scripts from the internet instead

It is a one-line script and not difficult to check and understand. And that line fixes (or at least works around) an issue the a vendor isn't willing to fix. I think we wouldn't have half of this discussion if by now there would be the understanding that the element devs would recognize that there are a lot of users out there that aren't living on bleeding edge and don't want to and have their own good reasons, and they are users that deserve support, too.

[t3chguy]

The devs understand this. We're still awaiting management to make the final decision here.

[andrewshadura]

What prevents you from reverting the PR which introduced this regression now, and then wait for the final decision?

[sumpfralle]

Just for an extra data point: I was curious, whether this (IMO: breaking) change stops people from keeping their instance of Element up-to-date.

(Personally, I needed to disable our automatic updates for the eight instances of Element I maintain, since many of our users suffer from the issue discussed here.)

Since I did not manage to find any kind of "collection of Element instance URLs" out there, I simply queried a search engine (here: duckduckgo.com) for a list a sites containing an Element-specific string:

1. search for an Element-specific string: https://duckduckgo.com/?t=ftsa&q=%22Sorry%2C+Element+requires+JavaScript+to+be+enabled%22
2. expand the list (click on "more results") until no further results are added
3. save this state of the website as an HTML file

This approach resulted in 175 URLs of Element instances for me. Due to search engines being hard to predict, this count may differ for others.

Then I analyzed the result and emitted the number of Element instances found for each discovered version:

cat search-result.html | sed 's|<a href=|\n<a href=|g' | grep 'rel="noopener" target="_self" data-testid="result-title-a"' | cut -d '"' -f 2 | cut -f 1 -d "?" | sed 's|/$||' | while read -r url; do version=$(curl -sSL "$url/version" 2>/dev/null | head -1 | grep "^1\." ); printf '%-20s\t%s\n' "$version" "$url"; done | sed 's/[^0-9.].*$//' | sort -V | uniq -c

The result is:

     28 
      1 1.7.21
      1 1.7.27
      1 1.9.0
      1 1.10.3
      3 1.10.4
      1 1.10.10
      1 1.10.13
      1 1.10.14
      1 1.11.0
      2 1.11.8
      1 1.11.13
      1 1.11.15
      1 1.11.17
      1 1.11.20
      2 1.11.28
      1 1.11.33
      1 1.11.35
      1 1.11.39
      5 1.11.40
      1 1.11.42
      1 1.11.43
      1 1.11.45
      1 1.11.48
      1 1.11.50
      2 1.11.51
      1 1.11.52
      1 1.11.53
      2 1.11.55
      6 1.11.57
      3 1.11.58
      4 1.11.59
      2 1.11.61
      2 1.11.62
      2 1.11.63
      2 1.11.64
     10 1.11.65
      9 1.11.66
      4 1.11.67
     10 1.11.68
     21 1.11.69
     10 1.11.70
     24 1.11.71

I interpret this as follows:

• Loads of people do not update their instances of Element automatically/periodically. Only 24 instances are up-to-date.
• The last version compatible with the discussed Firefox ESR version (1.11.69) is deployed more often, than the two versions around: one older version and one newer version are both deployed only 10 times (compared to 21). Thus, maybe 11 out of these 21 Element instances are manually prevented from updating (maybe due to the issue discussed here).

In my understanding, this could be understood as:

• 35 instances of Element are updated automatically/periodically
  • 24 running at 1.11.71
  • 11 statistically "unexpected" deployments of 1.11.69
• 11 of these instances are currently not updated automatically/periodically anymore

Summary: around a third of the well-maintained instances of Element out there disabled/skipped their automatic/periodic upgrade procedures. This does not feel like a healthy situation for me.

Disclaimer: the numbers above obviously rely on a tiny fraction of all world-wide instances of Element (a lot of (semi-) private instances (including mine) are missing). Thus, these numbers may merely serve as an indication of the state - not as any kind of proof.

[t3chguy]

What prevents you from reverting the PR which introduced this regression now, and then wait for the final decision?

Reverting two prs and doing releases on four projects will eat up most of a day. Can't do that without management signing off on that time.