Adding ruby_dep breaks semver

listen 3.1 breaks on older rubies now where 3.0 used to work. This change should have been done across a major version release.

Check the issue threads (some are closed). I'm on my phone, so I can find links for you.

Basically, it isn't breaking SemVer at all. It's a long story, so either trust me on this, or research the issue threads if you're really interested on the gory details.

If you value your time, just upgrade Ruby.

This change has had an insane amount if thinking and discussion behind it.

In short: everything is perfectly fine. Upgrade Ruby or upgrade nothing.

I understand your worries extremely well, trust me on that. It isn't like you think at all. (I'm on my phone, so no room to explain).

Check issues (recently closed too), changelogs, commits, wiki updates for Guard, Listen and ruby_dep. You should get explanation ls and answers there. If not, open an issue in ruby_dep.

There a ton of false beliefs about SemVer and versioning, so please approach this with an open mind. On May 4, 2016 10:31 PM, "Chris Eppstein" notifications@github.com wrote:

listen 3.1 breaks on older rubies now where 3.0 used to work. This change should have been done across a major version release.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/guard/listen/issues/390

I am a maintainer of Sass. It uses listen. Our users are not ruby developers as such, we have a very conservative policy about supported ruby versions because most of our users are using the system install of Mac OSX ruby.

So upgrading ruby isn't something I can do and it's not something they can do.

I understand that you think this is a problem with rubygems. I agree. But even if they fixed it, upgrading rubygems isn't something our users can do or should be expected to do.

I'm not going to get into a semantic debate with you about whether semver was violated. Installations are breaking unexpectedly. CI tests are failing.

I understand that you didn't sign up to support users of legacy ruby versions and that we did. I think the best option is just for me to fork listen 3.0.x and if there's any important fixes in the future, to back port them.

You can simply ask for backports if you need any. It doesn't make sense to keep new features working with older rubies.

(sigh)

I'll be blunt because I'm on the phone: you don't understand the subject, so your missing the point.

And the point is: ruby-core team doesn't support Ruby < 2.2. So it doesn't make sense to mindlessly upgrade Listen beyond patch level fixes if you can't even upgrade Ruby on a patch level basis. It's a packager problem and the debates and pressure should be there and not here.

I don't need to release a major version with no changed Api and no changed docs, just to keep doing the same ridiculous thing everyone else is doing: making sure every minor Api change is tested on 11 (!) Ruby versions (most obsolete and unsupported) on 4 (!) different operating systems and 3 modes of backend drivers and about 3 configuration options (more coming).

If you multiply that, you get 396 Travis builds, which take 5-10 minutes each and so many system resources in a brittle way, that they can't be run simultaneously.

Testing them all would take a dedicated serv:er, or a massive library rewrite (planned ahead).

All because you're relying on SemVer to make minor version updates (instead of patchlevel, like all security-level update situations require) just so users can run newer versions of Listen with API updates they don't need on Ruby versions that are not only permanently buggy (meaning: a single constantly triggered Ruby bug or crash due to a system update can suddenly force users to upgrade their Ruby anyway) but also allowing them to use a downright dangerously vulnerable version which can get their systems hacked.

You're a reasonable, respectable person perpetuating the silly popular idea that a single codebase should work on multiple, incompatible versions of a interpreter (which is arguably fully SemVer compatible).

The idea is silly, because I've never rejected a backport request, and I respond to bugs reported with version releases. So even maintaining your own fork makes no sense if your just going to use it for backporting.

You're also making the very risky assumption that the versions of Listen that worked before we're actually being tested. According to the Readme, Listen 3.x isn't compatible with Ruby 1.9.3 (and documentation is part of the Api). Unfortunately, the actual Ruby dependency wasn't updated.

So by using Listen 3.x, you've already dropped support for Ruby 1.9.3. Otherwise, you should've stayed on Listen 2.x.

If you don't want to support 2 Listen versions, then that's a double standard.

If you're a sass maintainer, know that I once proposed listen-compat to guarantee basic support for all future Listen versions regardless of Api changes and implemented it to work with sass. Even with sass's vendored Listen. Result? All my work was rejected for some ridiculous reason. ("too complex" if I remember correctly, so I have more reasons to be passed of than you are - and yet I'm not ).

My request: don't spread FUD unless you really know what's going on. Breaking a Travis build does not imply breaking SemVer.

In fact, relying on Travis as "proof of interpreter compatibility" is unreasonable.

In fact, thanks to this "breakage" you at least know that Listen 3.x isn't supported on Ruby 1.9.3 and you can do something about it to mitigate any current or future issues.

A "broken Travis build" is not a use case sass users are experiencing. If you do have one like that, let me know immediately. If users can't upgrade Ruby, they should downgrade Listen.

I take extreme care in making sure new functionality is robust and that users either have an trivial documented upgrade path, or they get a quick workaround or fix for their unique problem.

However, supporting multiple unsupported Ruby versions is just a crazy idea for a non-trivial library like Listen. So is confusing users with releases from multiple forks when the original author already treats security patches with insane seriousness and puts in less roadblocks for backporting than the sass project did by comparison.

On May 5, 2016 1:18 AM, "Chris Eppstein" notifications@github.com wrote:

Closed #390.

— You are receiving this because you commented.

Reply to this email directly or view it on GitHub

You can simply ask for backports if you need any.

That is great to hear.

It doesn't make sense to keep new features working with older rubies.

I never asked for that.

you don't understand the subject, so your (sic) missing the point.

No, you're missing the point. I am not fighting you here. But you are being very dismissive of me and I find the hostility of your reply disconcerting and unwarranted.

My request: don't spread FUD unless you really know what's going on.

I'm not spreading FUD. I opened an issue because I made no changes to my software but it broke using a semver dependency on listen (~> 3.0). I didn't take to twitter to whine and I didn't make any accusations against you nor did I use the kind of belittling language that you have used with me.

We simply have different sets of maintenance needs and expectations. I don't expect you to treat developers of ruby projects with the sort of kit gloves we we treat sass users with.

By releasing a fork, I can verify that updates released on that gem name won't cause problems for our users due to the issues that rubygems should handle but doesn't (namely: the current ruby version as a resolution constraint). That isn't something you think you should care about, and I respect that. It's something that I do need to care about given the needs our user base. As I've explained, what you consider "a trivial documented upgrade path", isn't trivial for our users because most are not ruby developers and do not work on ruby projects.

A "broken Travis build" is not a use case sass users are experiencing.

Thankfully. This CI failure alerted us to an issue that would have been experienced by users had we already released that version of the code.

If you do have one like that, let me know immediately.

This won't happen now, because I will depend on our fork so we can test things according to our use cases before they go out.

If users can't upgrade Ruby, they should downgrade Listen.

My point is that downgrading ruby libraries isn't something they know how to do and as much as we've tried to document such things over the years, we've seen that this sort of thing causes our users much heartache. They don't use bundler and global ruby dependencies have a tendency to repeatedly re-install.

So is confusing users with releases from multiple forks

We're not publicising this. Our users don't need to know or care about our library dependencies. You've stated quite clearly that the use case of support that our users have is not a concern for you, so the only way for us to ensure a pain-free install of Sass is to add a QA step before we release our fork which will track against the v3.0 branch that you are carefully maintaining so far and for which we are grateful.

when the original author already treats security patches with insane seriousness.

I've never questioned that. We are removing our vendored version of listen in Sass 3.5 (this work is already done) and we dropped support of ruby < 2.0 in our next release (currently deprecated in stable) so that we could do that. We were looking forward to having a normal gem dependency on listen. I would love to simply depend on ~> 3.0.7 and be done with things but it's not clear to me at all that you will even feel a need to backport security releases for as long as we will continue to need to support ruby < 2.2 because of how OS X manages the system install of ruby. This is simply a precaution I am taking now so that you will not be burdened by a set of users that you do not consider to be your target user.

But you are being very dismissive of me and I find the hostility of your reply disconcerting and unwarranted.

Thanks for letting me know. My bad. :(

I didn't take to twitter to whine and I didn't make any accusations against you nor did I use the kind of belittling language that you have used with me.

Again, my bad. I'm looking for a way to be brief, clear and polite all at the same time. I'm overreacting because I'm discouraged with all the upfront assumptions I get. No clear "I have problem X", but you started with "you broke X". That's accusatory and belittling to me, which is probably why I responded likewise.

Just read your opening sentence again:

listen 3.1 breaks on older rubies now where 3.0 used to work. This change should have been done across a major version release.

You're assuming (incorrectly in my opinion):

That Listen breaks something.
That Listen breaks something on older rubies.
You're assuming Listen 3.0 used to work.
You're assuming this change needs a major version released.
You're assuming the problem is with the change.
You're assuming Listen should be fixed (and not something else)
No mention of you're actual problem.
You made no mention that those are all assumptions.
No details for me to refute issue-by-issue.

All that in just 2 sentences. So I was anticipating "another" SemVer debate here. That's why I got discouraged.

I'm not spreading FUD.

All your assumptions are wrong from my perspective. You didn't ask a single why, so you assumed you're correct and that I'm wrong.

I'm wrong to say you're spreading it. You might just be a victim. Or you just phrased what you wanted to say in a way I misunderstood your intention.

I don't expect you to treat developers of ruby projects with the sort of kit gloves we we treat sass users with.

Which is why I expected some specific case/problem here from a developer like you. If you were a user, I'd anticipate a back-and-forth, step by step, I'd ask you for your Guardfile, your ruby version, your listen version, etc. Just like I would for any other user. Then I'd work out what is exactly the use case, and I'd go from there. I'm very polite to beginners. For developers who have "made up their mind I screwed up" - I don't have that much patience. You came across as that kind of developer. You definitely didn't start open-minded here. (And neither did I to be honest).

By releasing a fork, I can verify that updates released on that gem name won't cause problems for our users due to the issues that rubygems should handle but doesn't (namely: the current ruby version as a resolution constraint).

You can also just lock to Listen 2.x or lock to Listen 3.x and ask for backports relaxing the version constraint. If you are very concerned about upgrades, you should lock to a patch-level, anyway. (And just watch for new Listen releases).

The point is: locking to a specific release of Listen is less confusing to everyone, unless you really want custom patches that are going to be rejected by the orignal gem author.

The only thing I'm refusing here is dragging Ruby < 2.2 support through the 3.x API line. That's why I released the fix for the bad version requirement as a minor (3.1) so 3.0 can be branched (which it was recently) to backport patches (which happened - for an OSX dependency workaround for rb-fsevent).

So in your case, you can just lock to '~> 3.0' and you'll get patches/backports there - but usually only upon request. (Unless there's a bad bug).

Overall, I'd say you expected to lock to the major to allow for minor changes (which doesn't make sense in your case). You're not just depending on the API (major), you're also depending on controlling dependencies (Ruby), so the only thing you should "allow" is patch-level updates.

As I've explained, what you consider "a trivial documented upgrade path", isn't trivial for our users because most are not ruby developers and do not work on ruby projects.

If it isn't trivial for your users (which I understand), the solution is lock to the minor version. Because it pretty much puts you in the same category as developers maintaining servers running on non-upgradeable Ruby versions. In that case they only need security updates and bugfixes.

It's about extremes: either you go for "the newest" or "minimum disruption". A balance here doesn't make sense (locking to minor version). Especially if you're not interested in new features. (I doubt Sass ever will). If you ever do, backporting would be considered.

This CI failure alerted us to an issue that would have been experienced by users had we already released that version of the code.

Which is why I see this as a zero-problem issue. If dependencies are so important for you, you really should be locking to minors - otherwise every Travis run you make can be potentially working on a different set of dependencies. You really need to lock to patch levels - I can't stress that enough. So the issue is: your own dependencies are suited for your situation, regardless if it's the Listen gem or something else.

So not only did you avoid an issue here, if you review other dependencies you have, you may find other gems where a similar SemVer-valid release can drop a Ruby dependency. And you may not find it in Travis, because that release may appear a few seconds later. The fact that whether a major is needed is debatable - which means it's not a technical rule you can rely on across projects. Because even if I reverted my change, someone else could've done the same.

But that probably doesn't matter if you're not locking to Listen at all. (Which I think is a problem to begin with).

If you want to lock to a given version of Listen (so users can install Sass without it) by just vendoring to an updated version, you might find https://github.com/e2/gem-checkout useful.

My point is that downgrading ruby libraries isn't something they know how to do and as much as we've tried to document such things over the years, we've seen that this sort of thing causes our users much heartache.

Then you need some kind of cross-platform tool for installing sass. Windows apps have installers, even for Ruby. And sass would need a "self-update" feature. Maybe you should have a package like "sass-easy", which, when installed, would change the behavior of the gem.

I would love to simply depend on ~> 3.0.7 and be done with things but it's not clear to me at all that you will even feel a need to backport security releases for as long as we will continue to need to support ruby < 2.2 because of how OS X manages the system install of ruby.

Security releases are likely extremely rare here, especially after dropping support for TCP in 3.x. My policy is to always make a security release available ASAP, before even opening an issue publicly. However, I don't spend time backporting that fix to every major and minor release to a project if it's in search of maintainers. My assumption is: people need to either upgrade or track and ask for backports.

This is because I already spend a ton of time helping beginners even if "trivial" problems. I think it's fair for me to focus on writing code no one else can, as opposed to backporting patches which people who decide not to upgrade should learn how to do. Of course, I can still decide to do it myself, but it's not a promise at all. (I have too many of those already).

Anyway, I apologize for my reaction and what I said. I assumed you were just going to complain and waste my time without giving me any useful feedback on what to improve or a use case.

These issues are difficult and complex, and I wish I knew how to communicate them in simple terms. Ironically, helping beginners takes a lot less time and emotional effort.

I would love to simply depend on ~> 3.0.7 and be done with things but it's not clear to me at all that you will even feel a need to backport security releases for as long as we will continue to need to support ruby < 2.2 because of how OS X manages the system install of ruby.

I feel the need, but I'm not going to make 15 attempted releases (I just counted) of Listen on every security or bug fix (because that's would "full support" would take). I'm assuming security aware-people will investigate and submit/ask for backports.

This is simply a precaution I am taking now so that you will not be burdened by a set of users that you do not consider to be your target user.

It's not that I don't consider them the target user - it's that they're the most expensive target user. Very expensive. So I just have to prioritize based on the resources I have. And the best is: prevention and timing. Given I don't have enough resources, it's best to "survive" with what's available (meaning cutting support to the essential minimum) so that more can be done to improve the codebase and prevent issues - so that even earlier Ruby versions can get support added later, when this becomes much easier and less expensive to test and support.

It's a chicken and problem - to have the resources for support, I need a better codebase and faster testing. And I can't do that if I spend all my resources on just support. (Or debates about semver or replying to angry people who have failing Travis builds and can't give me a single practical example of how this impacts users in the real world - debates, where I'm patronizing for breaking SemVer and "threats" such as "I'll stop using guard because breakage like this pisses me off").

There's a bigger danger in Ruby versions. E.g. the recent JRuby fixes were because the Ruby API of JRuby and MRI are simply different. Maybe as a bug, but they're different. And if they get fixed, it's hard to even classify whether it's a bugfix or an API change. JRuby functionality was actually somewhat broken until just a few days ago.

I admire that you put in so much effort to accommodate casual users. I do wish more time was spent not debating with me over support for obsolete Ruby versions, but actually solving the root issue here: making Ruby easier to install/upgrade for casual users. That's something Listen can't solve, no matter how many changes I revert or what dependencies I set.

Let me know if I can help in any way. Please don't assume solutions and fault up front in the future. All because you're close to end users doesn't mean you have to act like one. I'm a developer, I understand failing tests, code and devops. So all I need is a specific issues to discuss and fix.

Especially if you have a situation so unique as with Sass (meaning: Listen is both a dependency and it can work without it), your issue should've been a lot different than the generic abstract "philosophical" complaints I get recently.

I've just relaxed RubyDep's internal dependency so it can be installed on OSX 10.10 because it's installed with an outdated Ruby 2.0.0.

While Listen can't be installed there anyway right now (it requires Ruby >= 2.2.3), RubyDep 1.3.0 is still an improvement, because at least the install error will be about Listen failing to install, and not RubyDep (which users are less likely to know about).

I've also added better warnings which show the current Ruby version and possible migration paths (including more conservative options). If you have any tips about what to show (or link to) to help users update Ruby to the version they want/need, open an issue in ruby_dep.

Just to give you a heads up, my plans for Listen are:

Fix misc bugs on related to crashes during closing (on rb-inotify) and JRuby issues (so you'll likely want those patches backported).
Extracting the backend drivers under a unified internal API inside Listen-core (a major change, though API should be 3.x compatible from SemVer perspective).
Probably create something like Listen-filesystem for the middleware layer and filesystem syncing
At this point the rest of the Listen codebase should be easy to stub, so I should be able to support more combinations of Ruby without an insane Travis "waiting time" between commits.
Symlink support
At this point, it will likely be performance optimizing patches released as minor version updates (in case I need to branch and switch implementations).
Restore network support (so people can work on VMs again).

A milestone will be after step 4, because it will allow implementing large directory support. (For users with huge projects who currently may complain about hangups or high CPU usage issues or unresponsiveness).

One key feature I want to implement is: detecting changes between Listen invocations, so that the user always has files properly regenerated, without needing to manually "trigger" changes on all files.

This may include using sqlite for a database, etc. I might use YAML or JSON or BSON at first, though.

So if users have any existing challenges with Listen, do send them to me for debugging.

Hope that helps you make good decisions without wasting time, etc.

guard / listen

Adding ruby_dep breaks semver #390