e2 / ruby_dep

Automatically helps determine supported Rubies based on .travis.yml file
MIT License
23 stars 7 forks source link

Can't install ruby_dep because ruby_dep itself needs Ruby >= 2.2.5 #20

Closed krisleech closed 8 years ago

krisleech commented 8 years ago

Locally I am using Ruby 2.2.4, however when I do a bundle install ruby_dep raises an error saying it needs ruby >= 2.2.5 because of the constraint in its own gemspec. which in turn reads .travis.yml.

This dog fooding, while cute, means I can't install ruby_dep locally on anything other than Ruby >= 2.2.5m forcing an upgrade.

I assumed setting export RUBY_DEP_GEM_SILENCE_WARNINGS=1 would work, but it appears not too.

Many thanks.

e2 commented 8 years ago

If your setup prevents Bundler from falling back to a supported version of the gem, lock to RubyDep '~> 1.3' and you should be fine.

That's if you can't upgrade for some reason.

Also, for dependency handling see here:

https://github.com/guard/listen/wiki/Ruby-version-requirements

If you can spare a moment, let me know why you can't upgrade (if you're stuck or something). I can relax the dependency if there's are real technical need.

https://www.ruby-lang.org/en/news/2016/04/26/ruby-2-2-5-released/

And the list of fixes and improvements:

http://svn.ruby-lang.org/repos/ruby/tags/v2_2_5/ChangeLog

E.g. previously, Ruby 2.2.4 was supported because JRuby wasn't released with a 2.2.5 equivalent.

Upgrading Ruby should ideally be as easy as upgrading gems like RubyDep. I get that on server farms upgrading Ruby may be expensive, though it shouldn't be more expensive than installing/updating multiple gems.

Whenever you're installing gems, you usually want either one of 2 things:

  1. Installing exact, known, working version locked by patch-level.
  2. Installing the latest possible gems for development, to reduce upgrading work when it is less expensive to do sooner.

I'm interested in your situation, because decisions like these are tough. No one wants to be forced to upgrade. Some people get angry even if they're forced to upgrade a version of Ruby that is vulnerable.

I'd rather work on helping you easily upgrade your Ruby version or help contribute to Bundler so upgrading is painless.

Sorry for the issues. I'm still wondering how to deal with this.

I think it's better for me to make a mistake that you can overcome, than for you to be forced to upgrade Ruby at a very inconvenient time due to a Ruby bug or vulnerability.

The Ruby 2.2.4 support was dropped because of JRuby 9.1.x series has been around already and 2.2.5 and 2.3 are usually already provided by Amazon and other hosting companies.

Thanks for letting me know.

On Fri, Aug 12, 2016 at 01:27:55AM -0700, Kris Leech wrote:

Locally I am using Ruby 2.2.4, however when I do a bundle install ruby_dep raises an error saying it needs ruby >= 2.2.5 because of the constraint in its own gemspec. which in turn reads .travis.yml.

This dog fooding means I can't install ruby_dep locally on anything other than Ruby >= 2.2.5m forcing an upgrade.

I assumed setting export RUBY_DEP_GEM_SILENCE_WARNINGS=1 would work, but it appears not too.

Many thanks.

You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/e2/ruby_dep/issues/20

dwt commented 8 years ago

OS X 10.10 (current version) still comes with ruby 2.0.0p648 (2015-12-16 revision 53162) [universal.x86_64-darwin15]. So all gems that carelessly depend on ruby_dep to set their version will initially pull in 1.4 when installed via bundler.

So I think it's kind of ironic that the tool to declare the dependencies has stricter version constraints that the original tools.

So please relax these constraints.

e2 commented 8 years ago

If OS X 10.10 has an outdated version of Ruby, I don't work at Apple to be able to change that.

People should install a maintained version of Ruby. I don't use OSX, so I don't know about the installation options (Brew?).

Updating Ruby should ideally be just as easy as updating a gem. Especially since Ruby more-or-less follows SemVer. (Except on the C-side and perhaps for bug fixes that need API changes).

Another thing is: newer versions of Bundler should automatically resolve to older versions of RubyDep and Listen.

If it's impossible to satisfy those dependencies, let me know.

Please support the ongoing work on Bundler, so that dependencies like these can be resolved automatically. (Try a pre-release version of Bundler).

There are also options to avoid the stock version of Ruby on any OS: rbenv, RVM, Brew, setting your own variables, etc.

You also probably don't need to upgrade Listen if you don't need to upgrade Ruby. Because it makes no sense to avoid Listen bugs (which I can backport to older versions if needed - just ask) and yet opt-into known Ruby interpreter bugs and vulnerabilities that will never be fixed.

I'm more than eager to relax the constraints if there's a specific important business case for it. E.g. outdated EC2 images on an important hosting provider, unable to upgrade Bundler, needed JRuby versions unaffected by vulnerabilities, etc. (But all of these have been already handled).

Relaxing the constraints doesn't seem to solve the real problem.

If Bundler can't install, it's a Bundler issue - unless Bundler cannot find a combination of gems to fit your constraints.

A better solution is to backport the fixes you need to an older version of Listen. But you have to tell me what you need, so I can carefully prepare a minimal-change release without breaking anything.

On Sun, Aug 21, 2016 at 11:08:09PM -0700, ‮rekcäH nitraM‮ wrote:

OS X 10.10 (current version) still comes with ruby 2.0.0p648 (2015-12-16 revision 53162) [universal.x86_64-darwin15]. I've been tripped now because the listen gem depends on this, and even though it installs fine on this ruby, through this dependency it is now tied to a newer ruby.

So I think it's kind of ironic that the tool to declare the dependencies has stricter version constraints that the original tools.

So please relax these constraints.

You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/e2/ruby_dep/issues/20#issuecomment-241322158

zetetic commented 8 years ago

Upgrading Ruby should ideally be as easy as upgrading gems like RubyDep

Well it isn't. Don't forget you are dealing with a wide range of users, some who are experts, some who are just starting out -- for those users even installing Ruby can be a challenge, let alone upgrading it. And even if upgrading Ruby were easy it isn't always something that people have time to do right at that moment.

I'm more than eager to relax the constraints if ...

Stop. Back up. IMHO the only reason for constraints is that they are actually required to make the thing work. If you think that people need to upgrade because you think they should -- for whatever reason, including security -- you have it backwards.

Making my (many) Ruby installations secure and bug-free is my problem. Let me handle it.

e2 commented 8 years ago

If installing or upgrading Ruby is a challenge on OSX, I can't help because I don't have or work on OSX to help fix that issue.

If someone has a cold and they're coughing, the solution isn't to just cover their mouth because "it's easier".

And just as people don't have time to upgrade Ruby (5 minutes?) I don't have time to spend weeks supporting and testing an outdated Ruby versions just because they don't have 5 minutes.

Most people don't understand the cost of compatibility - codebases end up complex, dead and no one wants to maintain them if there's a burden that using X "won't work on an older version of Ruby".

Overall, there are many reasons for not supporting older Rubies - mostly because it's VERY expensive in Listen to GUARANTEE compatibility with non-current Ruby versions.

That word "GUARANTEE" is important for some people. And because it's important for some, I can't make it so they have what they want and you're not affected.

Maybe frontend web developers don't care, because if something doesn't work they can just report/complain in the issues (which I would have to find time to respond to - as opposed to doing more useful work).

By reducing the supported Ruby versions I can continue to maintain and quickly respond to issues people have with gems like Listen.

Trust me - until you are a maintainer of projects like Listen, then Ruby versions are NOT your problem. They're mine, whether I want them or not.

Listen needs another rewrite - except that I don't want to support Ruby 1.8 syntax (too much work, too much frustration and the solution is upgrading Ruby). Not supporting Ruby 1.8 is fair. Dropping support for 1.9 is also fair. Turns out ... dropping support for < 2.2.5 is also fair due to security issues.

Turns out ... dropping support for 2.2.x would be beneficial, but 2.2.x is still officially supported.

And it turns out - there are no compelling reasons to support older Rubies.

If there are bugs in Listen due to threading issues in a given Ruby version ... you're not the one who is going to get notifications about bug reports.

For Listen to support any Ruby version, I'd have to support EVERY version.

And that's not fair given my limited time, especially since there's no payoff neither for me nor anyone else.

The solution here is to make a friendlier Ruby installer/upgrader for OSX. I don't see how relaxing constraints here helps such a project come about.

Also, spending 5 minutes upgrading is better than spending 5 hours debugging a known Ruby bug fixed by an upgrade.

Meanwhile, people are going to have to upgrade their Ruby anyway (at some point). So it's better to find 5 minutes now than to be forced to upgrade at a much more inconvenient time.

Again, people can choose to NOT upgrade Listen and/or RubyDep.

In short: the constraints are saying: "these are the supported versions of Ruby - using any other version is too risky for production".

I'm not forcing anyone to upgrade RubyDep or Listen. It's a choice, just like upgrading Ruby or not.

What you're effectively asking for is: for me to spend extra time to make sure things work on an older version of Ruby.

And I don't see a good reason to do that. I offer backporting and quick releases upon request (in case someone is seriously stuck with an older version and not viable way to upgrade their cluster or something - and in that case I want to know the exact bug, so I can make sure it's well tested to avoid adding to the frustration).

Summary: relaxing constraints isn't a good solution to anything (you can prove me I'm wrong based on a specific use case). It just fools users into thinking their Ruby version is supported when it officially isn't. And THAT can cause a lot more grief than the "inconvenience" of not being able to upgrade Listen without upgrading Ruby.

Solutions:

  1. Bundler should fall back to a version of RubyDep+Listen to match your Ruby version. Try upgrading Bundler if this doesn't work. If this still doesn't work, try upgrading to a pre-release of Bundler. If that still doesn't work, help out in the Bundler project - there's a fantastic team that's working on that feature right now.
  2. Upgrade Ruby. If you have time to upgrade Listen/RubyDep, then it's also the best time to upgrade Ruby. The sooner, the better. Especially with the bugfixes, speed improvements, syntax improvements, etc.
  3. Lock to an older version of Listen which doesn't rely on RubyDep.
  4. Ask for backports if you're stuck with a given version of Ruby and there's a bug you need fixed in Listen. This is probably want you want if you want to deal with vulnerabilities and bugs yourself.
  5. Provide a specific real-life business scenario where you're stuck and how none of the above are acceptable.
  6. Release your own version of Listen/Rails without the check/dependency. Yes, it's troublesome, but if the above solutions don't fit you, your situation is already incredibly demanding.
  7. Support the Listen project in some way so that older versions of Ruby stay supported. I don't recommend this option at all - it's not worth the attention.

On Fri, Aug 26, 2016 at 02:55:54PM -0700, Tom Head wrote:

Upgrading Ruby should ideally be as easy as upgrading gems like RubyDep

Well it isn't. Don't forget you are dealing with a wide range of users, some who are experts, some who are just starting out -- for those users even installing Ruby can be a challenge, let alone upgrading it. And even if upgrading Ruby were easy it isn't always something that people have time to do right at that moment.

I'm more than eager to relax the constraints if ...

Stop. Back up. IMHO the only reason for constraints is that they are actually required to make the thing work. If you think that people need to upgrade because you think they should -- for whatever reason, including security -- you have it backwards.

Making my (many) Ruby installations secure and bug-free is my problem. Let me handle it.

You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/e2/ruby_dep/issues/20#issuecomment-242861522

zetetic commented 8 years ago

@e2 that is a well-considered, thoughtful argument, and I appreciate the work you put in to it. There are only a couple of points where I disagree:

dropping support for < 2.2.5 is also fair due to security issues.

How so? If there is no other reason (like workload), I don't agree that this makes sense. Basically you are deciding that anyone who uses the gem should agree with your judgment vis-a-vis security. That smacks of paternalism.

I don't think any user would blame you for a security problem that they brought on themselves. You could certainly recommend that users upgrade their Ruby version, but to make it a requirement is just annoying.

5 minutes

5 minutes to install Ruby? Sure, on my development system, under ideal circumstances. It's not that it's hard to do, its that the decision to move up to a later version is a whole lot more complicated than how long it takes to install on one machine. We're dealing with multiple systems, multiple platforms, multiple projects, all of which have to be tested with a new version before we are ready to make that jump. It isn't just this gem --- open source projects move at different speeds, and we may have to hold back until other gems are working.

I appreciate your hard work in building open source software. We probably don't say that enough.

e2 commented 8 years ago

Thanks - much appreciated.

It's not about paternalism. It's about resolving the conflict between doing what the experts expect (no one has time to track thousands of vulnerability mailing lists) vs not making things harder for users.

Paternalism would be if I didn't give people a way out.

Here, no one is forcing anyone to upgrade Listen. If someone decides they want to keep a vulnerable Ruby, it makes sense to also keep an outdated buggy or even perfectly adequate version of Listen.

Wanting the newest Listen and not wanting the newest Ruby - that's an unreasonable demand. Demands like that take a huge toll on contributors, who spend sometimes hundreds of hours supporting a Ruby only because some business can't afford to find 20 minutes to test and deploy a cluster of 5 servers.

Mostly it's unfair to the Ruby core developers - they spend an insane amount of time investigating and fixing bugs, while also developing lots of valuable features.

A huge amount of time is spent for backward compatibility. The upgrade from Ruby 1.8 to Ruby 1.9 is probably a thousand times more challenging than from Ruby 1.9 to Ruby 2.4.

I wouldn't call it a "jump", it's more like "agrophobia". Fear of being bitten by issues while not being terrified of the existing old bugs and waste in productivity and opportunities. (And an upgrade will be inevitable anyway, so why not get it over with sooner and reduce the business risk down the road?).

(That's because of all the time put into decision making - just check out the ruby-dev mailing list or the issue tracker or the change history between versions).

Every time an expert or business upgrades to a newer Ruby, they risk maybe 10 new problems (and maybe 8 occur immediately and are straightforward to fix), but they save themselves thousands of hours dealing with headaches - now and in the future.

So a Ruby 2.4.0 project will likely support a Ruby 1.9 project pretty well (given the years and unthinkable amounts of hours spent on fixes, improvements and new features).

I'd say the only reason not to use the main development branch of Ruby is: sometimes it exposes too many bugs (or too many cases of poor coding) in existing projects. And not all of them are high priority (and it may take time until gem authors catch up).

But for every released version of Ruby, it's a no-brainer. Less bugs, faster tests, catching code smells sooner (while the person who created them is still hired and available to fix it), productivity increases (new Ruby warnings, frozen string literals, immutability, better APIs, new APIs).

It's actually a major issue if gems don't work on a newer released Ruby. So my policy is: lightning-fast releases whenever possible.

I believe that if a business can't adapt quickly, it won't survive long. This includes upgrading Ruby.

The only case for not upgrading Ruby is: if it affects time-to-market. And in that case, upgrading Listen doesn't make sense either.

This means: if some gems aren't keeping up right now, it's only going to get worse in the future. (So it's better to support them to move on to newer Ruby versions, so new maintainers can step in and everyone wins in the end).

The reason to make it a requirement is: so new Ruby features can be used in Listen. Not every new Ruby feature can be easily reimplemented for an older version, and the maintenance costs of doing so are huge.

I can't count the hours and frustrations from accidentally using a JSON-style hash syntax or splat argument syntax on code that was supposed to be Ruby 1.8 compatible. (Result: broken builds, crashes during run time, hours of rebasing, amending, squashing, commenting, dealing with duplicate issue reports, waiting for tests to finish, delayed releases, etc - all 100% wasted effort with no benefit to anyone).

Also, unless there's a requirement up front, I may get lots of bug reports about crashes on Ruby 1.8, etc. Many users are web developers who accidentally use officially incompatible Ruby installations and "syntax error" isn't instantly obvious to them that their Ruby is outdated. (Or that they're accidentally running a stock version of Ruby). And that means the issue tracker gets polluted with irrelevant bugs, duplicates, lots of email notifications, discourages users from searching the issue tracker, interruptions, wasted time, etc.

Meanwhile on servers, you won't see it if it's just "a warning" - and that's where it's better to crash on startup than 5 hours after people have gone home to their families after launching.

So instead of quietly letting Listen potentially fail during startup on an important server cluster, the "requirement" is really saying "if your Ruby version doesn't match, you have to consciously disable the requirement and test it properly yourself, because I can't guarantee it will work".

Backward compatibility often means that everyone is terrified to change a single line of code. Then it soon takes top experts to fix a bug without breaking the world. That's how projects (and whole frameworks) die.

And top experts have zero time nowadays. So little time, that they get annoyed even if it means upgrading their Ruby, no matter how beneficial and inevitable that is anyway.

So annoying you is not something I want, but I can't think of a solution with less downside for everyone.

I would really rather spend time helping you automate your deployments with e.g. Docker and automated acceptance tests, so that you can both quickly test a Ruby upgrade ... and ... deploy it just as fast.

I'd also rather spend time upgrading and maintaining other gems you have issues with - than allowing installs on outdated, unsupported Rubies.

If testing really is that expensive, you could consider just bundling a given Ruby version with your software. (Sounds insance, but it has lots of surprising upsides as well).

On Mon, Aug 29, 2016 at 04:43:45PM -0700, Tom Head wrote:

@e2 that is a well-considered, thoughtful argument, and I appreciate the work you put in to it. There are only a couple of points where I disagree:

dropping support for < 2.2.5 is also fair due to security issues.

How so? If there is no other reason (like workload), I don't agree that this makes sense. Basically you are deciding that anyone who uses the gem should agree with your judgment vis-a-vis security. That smacks of paternalism.

I don't think any user would blame you for a security problem that they brought on themselves. You could certainly recommend that users upgrade their Ruby version, but to make it a requirement is just annoying.

5 minutes

5 minutes to install Ruby? Sure, on my development system, under ideal circumstances. It's not that it's hard to do, its that the decision to move up to a later version is a whole lot more complicated than how long it takes to install on one machine. We're dealing with multiple systems, multiple platforms, multiple projects, all of which have to be tested with a new version before we are ready to make that jump. It isn't just this gem --- open source projects move at different speeds, and we may have to hold back until other gems are working.

I appreciate your hard work in building open source software. We probably don't say that enough.

You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/e2/ruby_dep/issues/20#issuecomment-243291226

Confusion commented 8 years ago

People should install a maintained version of Ruby.

We would love to install a newer version of Ruby. Unfortunately there isn't a released rvm version that enables us to deploy 2.2.5 to our machines. So because this project and the most popular ruby version manager are out of step, we're now unable to run bundle update to keep our dependencies up to date. That is kind of a counterproductive result: we want to be up-to-date with at least our gems, but can't be.

e2 commented 8 years ago

Unfortunately there isn't a released rvm version that enables us to deploy 2.2.5 to our machines.

If the RVM release is behind, you can fork RVM, branch it and install from your branch,e .g.

rvm get branch Confusion/my_branch

Confusion commented 8 years ago

Sure, and apart from me, all users of gems depending on ruby_dep would have to do that, resulting in a lot of aggregate effort. On top of that, it's hard for all of them to judge whether perhaps there is a good reason (open bug in one of the last commits) for the code not to have been released yet. So I don't think that solution quite works.

e2 commented 8 years ago

@Confusion - good points, thanks.

_TL;DR - I didn't want to suggest forking is easy and effort-free ... I'm saying that may be the only option left: for some one to either find a way to help that project, or heroically take things into their own hands. _

Ruby 2.2.5 has been released 5.5 months ago. If RVM can't support it, obviously there aren't enough maintainers available there. So maybe that project needs help.

If a project isn't responsive (or crushed with issues/work/etc.), that's be best thing we can do: fork and take on the "ungrateful" role of maintainers. It's like everyone is silently waiting for someone else to pick up the project and help out.

I have spent time debugging RVM and solving all kinds of horrible shell/path issues (which sometimes weren't RVM's fault at all), but I was ok doing that and glad that I didn't have to be an RVM maintainer.

Nowadays there's more value in having contributors as opposed to users. Even opening up issues is a big thing.

Sometimes the most valuable contribution is helping get to the core of the problem (no matter which gem it ends up being). Many "easy" issues are actually complex and tough to solve elegantly.

I rely on issues to know people are having problems. Because for every issue reported, there are at least tens of users who won't bother to report, but will complain wherever else possible.

Sidenote: The RVM author was badly burned out at some point and had to step down.

e2 commented 8 years ago

TL;DR:

  1. I've just reworked the README to help solve issues: https://github.com/e2/ruby_dep/blob/master/README.md
  2. I can support 2.2.4 if there's a clear issue installing 2.2.5 and locking versions is expensive for someone. Please open a new issue with the use case if so.

Thanks to everyone here for the feedback. It obviously helped me a lot!

@krisleech - thanks for reporting this. The reason 2.2.4 was dropped is that I couldn't think of a scenario where 2.2.4 was necessary, given the list of bugs fixed in 2.2.5. It's not a massive upgrade in terms of features, so it can only "work better".

2.2.5 installation problems in RVM is something I didn't know about. Can someone point me to an issue there?

It seems silly that RubyDep doesn't support < 2.2.5, but with Ruby 2.4 being developed it just shows that there will be 3 minor-level versions of Ruby in play out there.

I am considering adding 2.2.4 support if there's a compelling reason (e.g. RVM issue) - so I know exactly when to drop support again.

On the flip side, I'm not going to bump the Ruby 2.2.5 requirement, unless there's a very strong reason there too. (I doubt there will be).

Confusion commented 8 years ago

It turns out I was wrong: I thought the output of rvm list known was authoritative, but it isn't. Although rvm doesn't list 2.2.5 as a known Ruby, it can install 2.2.5 just fine, because it is available in the 'expected location'.