mlugg
commented
2 months ago Hi, I'm coming out of my cave to give my opinion here. It should not carry as much weight as it would have in the past -- the decision here is ultimately up to AMJ and NeKz. Also, my responses here will probably be out-of-order and generally a bit all over the place -- sorry about that.
The versioning of SAR is based on semantic versioning
This is not, and never has been, true. If SAR tried to follow semver, almost every release would be major, and we'd probably have hit 50.0.0 by now: SAR has enough quirks that most non-trivial releases have breaking changes of some kind. An a.b.c format does not equal semver. Slightly weird analogy, but our versioning system is more akin to something like Minecraft -- it's more like 1.major.minor, but major and minor are based off of... vibes, to be honest.
This system, I will admit, is weird and ill-defined. However, that's... really not a problem? Semver is useful and important for software where backwards compatibility is important. For instance, a hypothetical Wormhole should follow semver (or a similar system) because mods rely on APIs exposed by specific Wormhole versions. SAR, on the other hand, has no such constraints, so going with a more "intuitive" system where the major version increments upon huge rehauls or changes of ownership, and minor increments on other releases, works pretty well IME.
Obviously yes, I know pre-releases are meant to actually implement the features in their initial states to be fixed/updated in feature pre-releases, but this current system puts no weight on full releases, making them feel like just a number changing for the same of game admins.
If you look at any piece of software with any kind of pre-release system, this will be, by design, how it works. Ideally, there would be zero code changes between the final prerelease in a release cycle and the following full release. Full releases are intended to be as battle-tested as possible, and come once bugs in a version -- particularly around important features like timing -- are mostly eliminated and the rate of new bug reports has fallen to near-enough zero.
I fail to see how this structure is a bad thing. The intention, as you seem to understand, is that users can use pre-releases at their own risk as needed; changes may be drastic or frequent, and bugs are to an extent expected. By the time a full release rolls around, the rate of changes should be effectively zero to guarantee stability and smooth upgrades for those tracking full releases.
When it comes to timing, full releases only happen once in a while [...] and pre-releases can come very sparatically (sic) [...]
The release schedule follows the rate of development. Some years back, when I was very active, I would work on SAR near enough every day of my life for months on end. This meant the rate of pre-releases was naturally much higher, because there was more active development, and hence more changes to dogfood. Nowadays, the rate of development has slowed. This is in large part because the current active maintainers simply do not have the time I did during COVID and the start of my degree: the amount of time I put into SAR was pretty close to a full-time job a lot of the time. I do not say this to boast about the effort I put into this project (I'm well aware it probably sounds like that), but rather to point out that it is unrealistic to expect SAR to maintain anywhere near that level of progress. So, it is to be expected that releases are much more gradual now than they once were.
As you later acknowledge, the people working on SAR are doing so on a volunteer basis, and for most of them, speedrunning is not a particularly major part of their life. Statements like "there may be days between any communication updates between devs", to me, imply an expectation of time dedication that is simply not realistic. Try comparing the release schedule of SAR to that of sister projects like SourcePauseTool, which will generally have one release every 4 months, despite having several more active developers (as you pointed out, the current SAR team is basically AMJ and NeKz).
Changing the versioning methods would also make the lives of leaderboard admins (hopefully) easier [...]
The current system exists as a reasonable safeguard against the high chance of a bug affecting timing or otherwise having a major impact on the game being introduced. A release schedule akin to what you have proposed would essentially just promote the majority of what are currently pre-releases to full releases. This is not easier for admins. There would be just as many SAR versions floating around; the difference is that subtle timing or gameplay issues are more likely to be introduced into a full release, and these become very problematic to deal with. You either have to allow a bunch of bad times, or roll back virtually every time from a given interval despite the runners in question following all stated rules. This sucks for all parties involved.
But sometimes I do really feel like even though work is being done, it's not being done in the places that people are wanting to see the most, or in the places that actually matter.
Broadly speaking, contributors to SAR are giving their time because they enjoy it: there are features they want to introduce for themselves or for others, and they are willing to put time into those. In my opinion, no member of the community who is not putting their money where their mouth is and actively contributing to SAR has any authority to dictate the direction of the project's development -- even if admins or the community at large broadly want a certain feature.
There is one area I will agree with you on: bugfixes, particularly those affecting previously-functional components of SAR, ought to result in pre-releases as soon as possible. Further, if the latest full release of SAR has regressed due to game updates, that is a good reason to get a full release out as soon as possible. If I had the time and energy to be involved in active SAR development today, I would want to ensure this is the case. However, if the current maintainers disagree, that is their prerogative. I am about to grant @ThisAMJ full privileges in the @p2sr organisation (I actually thought I'd already given him release powers here, so apologies if I've been indirectly responsible for any frustration there) so he will have the power to make releases as he sees fit.
ThisAMJ
NeKzor
JaioCG
Alex-Marek
I feel it's not an unpopular opinion to say that the current SAR release system has some major flaws to it, with the current "stable" release (
1.12.8) broken in multiple ways (quitting being broken, solocoop being broken, etc.), and any time the broken-ness of the system its mentioned, it's always quickly shut down with a "Do things yourself, bucko", which at least to me is pretty Royally toxic. Hopefully making a full issue about it can put some eyes onto making changes not just to how SAR updates are made, but to the entire release system in gemeral.The Current Release System
"Stable" Releases vs. Pre-Releases
SAR in its current state is broken up into two main versions, the main "stable" release, and pre-releases. As of writing this, the current "stable" release is
1.12.8, and the latest pre-release is1.13.0-pre4. "Stable" releases are relatively rare, happening only say once per year, while pre-releases are treated like main releases, only with the thought that if a feature is broken or illegitimate, then runs using the pre-release can be banned.With pre-releases effectively functioning as the only kind of release, large feature-set releases (like
1.13.0-pre4) and single-line literal crash fixes (like1.12.8-pre6) are treated the exact same. This causes issues when there is a crash or any other game breaking issue that needs solving, it could be pushed off for release just to add in some extra features (like the upcoming1.13.0-pre5fixing the +leaderboard issue)."Stable" releases (I put "stable" in quotes because
1.12.8is very much not stable lol) really only happen when SR.C and CM admins bug SAR devs enough to get a full release, due to the rules on legal SAR versions. Because of this, the "stable"1.12.8release only really released with some extra TAS features and fixes, while the pre-releases added a whole lot more (that wasn't adjusted for the full release) and fixed multiple crashes. Obviously yes, I know pre-releases are meant to actually implement the features in their initial states to be fixed/updated in feature pre-releases, but this current system puts no weight on full releases, making them feel like just a number changing for the same of game admins. I suspect1.13.0's full release will be similar, with the pre-releases having features like viewmodel shadow removing, Portal Reloaded co-op timing, new conds, co-op ghosts support, proper mod support on Linux, skipping workshop maps, multiple mtrigger changes, and more being fully fleshed out and finished, while the actual full release, or even any future pre-releases won't get much changes of what's before, essentially making those versions stable compared to the new.A little side note about the "forbidden gamemode" or whatever. I know cmm is probably planning to go into beta somewhere around
1.13.0(which I think was one of the reasons the minor value in the version was changed to 13), but at this current rate, a major feature addition like cmm won't feel any different to an update that only adds tiny changes, as they're treated identically as pre-releases.Wait... How do Releases Work, Anyway?
The versioning of SAR is based on semantic versioning (but not as fully-set), following a
MAJOR.MINOR.PATCHsystem. But currently, only the "PATCH" and extended pre-release version numbers are used, with MAJOR being stuck at 1 when there have definitely been updates that qualify for major version changes (speedrun timer rewrite, config+, overlay system rewrite for text/ghosts, etc.), and MINOR being stuck at 12 for multiple years, which is currently associated with an "era" of SAR's development, not based on new smaller-scale features and commands.When it comes to timing, full releases only happen once in a while (the period between
1.12.7and1.12.8's release was approximately 1 year and 2 months), and pre-releases can come very sparatically, some in quick succession while others can have months between them. I think the sporadicness is part of the reason why quick fix updates are delayed to add in more features, as there may not be a chance for those new features to be released in any quick time window, and may sit in PR hell for months (I'm looking at you, Reloaded co-op timing). Obviously this is not ideal, but if every update is treated as a big feature-set update, then there may not be any hurry to do a release when the feature list is small, even if it is. As of right now, NeKz is really the only one doing releases, and with the sporadic nature of a busy schedule, there may be days between any communication updates between devs or runners testing out fixes for releases, which I believe is also hurting scheduling. A recent example of this being the +leaderboard changes in1.13.0-pre4being quickly fixed by AMJ, but without release permissions, being stuck in limbo while runners are being told to "just downgrade".New Release System Proposition
I think the crux of the issue is that feature-set releases are treated identically to quick patches, both being labled as pre-releases, which can be seen with some of the updates I named above. Instead of actually using the MINOR.PATCH system that semver implements, or at least loosely trying to be similar, everything is dropped into a pre-release until SR.C or CM admins complain enough for a full release. I'm feeling safe with proposing a change here to fit more into standard versioning, hoping changes can be laid out easily.
Woah! Versioning That Makes Sense!
Start simply with releasing the current
1.13.0as2.0.0. I don't really have a reasoning for this, but it feels justified enough, and makes the rest of this new system work way easier. From there, MINOR versions can be used for feature-set releases, like1.13.0-pre4, while patches are saved for neccesary quick fixes, like1.12.8-pre6. These changes remove any stress of having to pack more features into what should be a quick bug-fixing release, as well as leaving pre-releases open to their original purpose.In my mind, the correct use for a pre-release would be only adding/releasing an in-beta feature, say cmm, letting players demo and test out the feature before properly merging it into a full release. This gives players the opportunity to test one specific feature, and the ability to fall back into a main release when not currently playing. This also justifies the "if X is illegitimate then we can ban the runs" policy, as these new pre-releases can be much more experimental compared to how they are now.
Changing the versioning methods would also make the lives of leaderboard admins (hopefully) easier, not having to deal with multiple different versions of the plugin, can just support the one current "main" version, and maybe a pre-release if one is available at the time. I say "if one is available" assuming that pre-releases will only be used for in-dev features, then bento a full release. essentially removed once the feature is merged i
Commonly Done Releases?
As well, this should fix any issues with release timing. If a release isn't expected to include many features, then it can get released as soon as possible, especially if more people have permissions to create releases. Sure, MAJOR versions should be way between one another, only being for major code rewrites or additions, and MINOR versions probably shouldn't be released daily. But to avoid issues where features or neccesary fixes are stuck in PR's or just merged without release, those things should be done as quickly as possible.
Ending Statements
I understand that everyone working on SAR, srconfigs, rules, mtriggers, and every other repo related to P2SR is doing it on their own volunteer schedule, which is, and should be, in the background of more important things (school, work, health). But sometimes I do really feel like even though work is being done, it's not being done in the places that people are wanting to see the most, or in the places that actually matter. I don't expect releases, features, etc. to be done on the daily, I think the current development speed and feature-addition speed of P2SR tools is perfectly fine where it is (even if I think some features should be focused on more than others), but it just feels like releases are slow, and only happen when absolutely 100% neccesary, and only if enough features are added.
Sorry that this issue is so long, being a massive information, suggestion, and complain dump, but this is just one of the issues I have with P2SR tools development that people just ignore, and that I feel needs attention brought to. Especially sorry if it does feel like I'm just complaining here, but it's just infuriating knowing that the people that do have power to change things, or at least the most vocal in the community, shut things down instantly, or try to hide/fake-delay new things existing when the community is curious and jsut wants some- hell, any- communication.
I don't think I'll make any more of these for now as this one was pretty tiring to write, trying to mix information I see written out on Discord with my own opinions, and suggestions for changes that won't piss too many people off. I really hope something about releases can change though, as I know it's one of many users' main issues with SAR, and P2SR development as a whole.