Closed aramprice closed 6 months ago
It seems more problematic than just installing Ruby early. Because you need Homebrew to install Ruby (at least with ram), but need a Ruby > 2.0.0 to install Homebrew.
The idea of an omnibus for Sprout has come up before. Perhaps this just tipped the need for this.
Aram, is this what you mean about Sprout bringing its own Ruby?
On Mon, 6 Feb 2017 at 3:57 pm, Davis W. Frank notifications@github.com wrote:
It seems more problematic than just installing Ruby early. Because you need Homebrew to install Ruby (at least with ram), but need a Ruby > 2.0.0 to install Homebrew.
— You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/pivotal-sprout/sprout-wrap/issues/99#issuecomment-277586905, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAfIrrHJMKdE1nsSVawsOb8mDanYx_-ks5rZqhNgaJpZM4LmHyB .
Here's what I did yesterday that worked, mostly:
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
soloistrc
, ensure that the homebrew recipes are presentcaffeinate ./sprout
This completed the sprout run without error. Most apps seem installed at least. And bash-it, etc., are present.
However, Homebrew complains that the Casks are in /opt/
. Not sure if/how to fix this.
I'm fine with this approach. I feel leaving Sprout is going to be a move backwards.
Regarding https://github.com/pivotal-sprout/sprout-wrap/issues/99#issuecomment-277586905 homebrew still/currently uses system ruby on macOS. Unfortunately this is ruby 2.0.0
:
→ /usr/bin/ruby -v
ruby 2.0.0p648 (2015-12-16 revision 53162) [universal.x86_64-darwin16]
If one is willing to double-bootstrap (first brew.sh, then sprout) then the recipe above works fine. This may be the way to go ultimately. If so then sprout
should stop trying to install brew see: chefcookbooks/homebrew. Currently sprout
assumes that it is responsible for installing brew
(and other formulae, casks, etc) so pre-installing brew
might cause problems.
Regarding Casks in /opt
it is likely that you have an outdated soloistrc
or outdated cookbooks. This isn't the behavior I'm seeing on an up-to-date sprout-wrap
.
Abandoning sprout after so much investment feels backwards, especially since it still works fine on new Ruby + Chef versions. Additionally, Brewfile
doesn't take into account things that are not in Homebrew, or otherwise hard to create Homebrew formulas for, making sprout-wrap
a much better tool for solving the problem that sprout set out to solve, as it gains the Homebrew library through the homebrew
cookbook while still harnessing the power of Chef for system configuration & provisioning.
After using many configuration & provisioning tools, I've seen that bootstrapping is one of the major problematic and difficult issues on any OS. This is especially true for Mac OSX due to the hard to automate dependencies such as XCode + xcode-command-line-tools, and the out of date system ruby.
So really, our problem is bootstrapping. This is a solvable problem, as myself, @infews, and others have managed to get from a fresh OSX install to a working one with RVM + Chef 12.x + soloist. So that means a path from fresh -> provisioned with sprout-wrap
exists. The hard part is going to be tracking down the right gem dependencies for a Gemfile.lock
, and managing any automation hiccups from fresh slate that come up during soloist provisioning. I've had to do a bit of that work in my fork, so it's available for reference here.
Just as with any configuration management & provisioning software, starting from scratch becomes important for testing to ensure missing dependencies are handled. Knowing this, I had originally intended to set up a test-kitchen
suite for running sprout cookbooks against a fresh VM for testing. However, with OSX this also became problematic due to the difficulty in finding Vagrant or VirtualBox VMs for Mac OSX (mostly it was licensing concerns that prevented people from publishing images, so you have make one yourself). I had some problems with VirtualBox + veewee, had minor success using VMWare Fusion, but then ran out of space & time to complete it. (disk space that is ;-)
So I still have yet to get a good test-kitchen
provisioning toolchain for Mac working properly. If we had that, finding the right "golden image" set of gems for bootstrapping on top of system ruby would be much easier. I haven't had time myself to learn & set up sprout-orchard
, but I'd imagine maybe someone at Pivotal knows. Maybe just starting with RVM + ruby-2.0.0p648
, creating a gemset & starting to pare down on the gem dependencies will get us 80-90% there.
Background: I've used sprout-wrap
myself at prior jobs & maintain a personal fork also for using it on a semi-frequent basis to manage & maintain my various Mac laptops. I also have a bootstrap script for XCode + CLT here. My attempts at getting an OSX VirtualBox VM created have been shared in the veewee
project here and here.
Update: So it looks like the chef/bento project has templates now for macosx-[7-12]
. These are probably worth a shot at using with test-kitchen
as they'll be much more likely to work out of the ".box
" (pun intended).
@trinitronx Thanks for that. Have there been any conversations since then? Is Pivotal actively using this tool? I just unboxed our first proper pairing station here at Periscope Data and I'd love to get us on Sprout-Wrap.
@ClayShentrup I made some changes to the Gemfile
and .travis.yml
which should ensure that sprout-wrap runs on the latest macOS.
Before adopting the soloistrc
contents of this repo though I'd suggest auditing the cookbook list. For example sprout-mysql
and sprout-postgresql
should probably be dumped in favor of brew
installing these (via homebrew: {formulas: ...}
in soloistrc
)
Thanks @aramprice! I'll give it a spin soon.
See https://github.com/pivotal-sprout/sprout-wrap/pull/102 for changes
Good news: macOS 10.13 beta 3 has a system Ruby of 2.3.3p222
As @wendorf points out, now that High Sierra's out we have:
$ ruby --version
ruby 2.3.3p222 (2016-11-21 revision 56859) [universal.x86_64-darwin17]
@trinitronx @wendorf good news is that un-pinning gems allows sprout to work on High Sierra (macOS 10.13) but unfortunately Travis CI doesn't support 10.13 as noted here.
@aramprice : Of course... d'oh! It looks like they have planned support for 10.13
soon:
This image still run on macOS 10.12 Sierra because we wanted to make Xcode 9.2 available as soon as possible. We will be releasing a new image with macOS 10.13 and Xcode 9.2 in the near future. Thank you for your patience!
Technically if sprout were using rvm
, it would be possible to use newer ruby version:
Ruby versions installed with rvm ruby-2.3.5 ruby-2.4.2 (default)
But of course, that defeats the purpose of testing the bootstrapping process using pre-existing system ruby.
Not sure if you guys have tested it yet, but it looks like Travis CI now supports macOS 10.13
!
I've been converting some of the cookbooks to use this, notably https://github.com/pivotal-sprout/sprout-base, https://github.com/pivotal-sprout/sprout-chruby, and https://github.com/pivotal-sprout/sprout-git
See https://github.com/search?q=org%3Apivotal-sprout+xcode9.3&type=Code
Ex: https://github.com/pivotal-sprout/sprout-base/blob/master/.travis.yml#L3
Many of the sprout-
repos should probably not be converted since their functionality can be accomplished by using brew
.
I mention this hear since sprout-wrap
references a number of them here and until they are either removed, or converted tests for this repo are unlikely to pass.
@aramprice: Thanks for pointing out the ones using the new Travis CI image! I'll have to look into adding some testing to my sprout-wrap
fork, mainly surrounding the ruby version & bootstrap parts.
I also need to get my fork up to date and test out the new sprout
script and it's own environment instead of system ruby or RVM (although both worked at various times). One of the more difficult things had been migrating ruby versions & getting the gem version dependencies right. This week I had the need to update dependencies and go through bootstrap again, so I had to make the time to do it.
Sounds good about migrating to brew
for most things, although it opens the door to less stability when important formulae or casks change. I've noticed over the years that some will change names unexpectedly, others get completely removed, and still others have to either be migrated to custom taps or build your own tap. I had to create a couple custom taps and chef recipes for a couple things that became deprecated by Homebrew. However, the vast majority of them keep the same name and stick around, especially if it's a more popular tool or app. Just be aware if that's the route you guys decide to go 😉
@trinitronx I may have spoken a tad too soon about the viability of the recipes noted above. I ported the changes I had from other repos to https://github.com/pivotal-sprout/sprout-wrap/pull/103 and the travis run appears to have succeeded:
I still think that nearly all of the recipes should be dropped in favor of a Brewfile
, or at the very least replaced with the homebrew
LWRP, but that's a discussion for another day.
FWIW we seem to be moving away from sprout mostly because folks have limited familiarity with chef and building enough context to understand it drives most folks to use bash
+ brew
. The indirection of sprout => soloist => chef => cookbook => brew tends to try folks patience.
Personally I'm experimenting with dotfiles and a Brewfile
@aramprice It's interesting to hear about that experience with "indirection". Do you feel similarly about this project from Labs Seattle? (Based on Ansible apparently.)
@ClayShentrup I haven't looked that closely at seattle-beach/alfalfa but I think there is a similar (slightly lower?) level of indirection.
For sprout at least we decided at one point that it would be macOS only. At that point there was no need for the multi-OS capabilities of chef. It's not clear that brew
was feature-full enough at that point (~2yrs ago?) to replace sprout-wrap/soloist/sprout-cookbooks/chef but I think that today brew
+ Brewfile
covers close to 80%-90% of what sprout-wrap/soloist/sprout-cookbooks/chef does with a fraction of the complexity.
Another lens to see this through: why should a developer invest in understanding a technology they wouldn't otherwise use (in this case chef) in order to automate something (brew
+ Brewfile
) that doesn't really (at this point) need it? This argument breaks down if developers live and breath chef (or some other tool). Most developers don't provision fleets of servers (chef's sweet-spot) so it has become an esoteric toolset.
@aramprice I agree. In fact, the last two times I rebuilt a macOS machine from scratch I bypassed Sprout-wrap entirely because I was having too much trouble wrestling with the recipes.
Hmm, interesting... hadn't realized Brewfile
was a thing now... definitely worth investigation. Although, sprout-wrap
+ soloist
still are very useful for those of us who have developed custom Chef cookbooks for OSX apps & software (like me). In fact, this project has safely migrated my macOS system setup across 3 separate work laptops, through 2 or 3 (lost count) OS upgrades, and a company acquisition! Definitely would have lost much more time if I didn't have sprout-wrap
! Great work to all involved in making this project a reality 🥇 👍 😄
Also, it's proven very useful to install DAW software + plugins and other things on my home laptop.
I wonder if Brewfile
and a bit of shell script glue could make the Chef bootstrap process easier by using the chefdk
Cask. Then, sprout-wrap
could use ChefDK + embedded Ruby to avoid inherent problems with system Ruby? So process would look like:
Brewfile
or other one-off command to brew cask install chef/chef/chefdk
/opt/chefdk
sprout-wrap
+ soloist
can now continue for custom macOS Chef cookbooks!Thanks for the success story (stories), it is great to hear about sprout automation paying off folks!
The chefdk
thing seems interesting. It sounds like a good path forward for folks who have an existing investment in sprout/chef cookbooks.
Based on the puzzled looks I get from folks when explaining chef cookbooks to folks though I suspect that it won't be enough to revitalize sprout-wrap
for the general population though :-/
I just had to provision a new MacBook Pro (15-inch, 2019, macOS 10.14.6
Mojave) for work... it turns out that after a couple hurdles with the new XCode + CLI tools, and the missing ruby.h
header files, I was able to successfully run soloist
on my sprout-wrap
fork!
This was starting out with system ruby too. I avoided installing rvm
until after getting a fully successful Chef run (also, sous-chefs/rvm cookbook is presently unstable). So as long as Apple still provides the macOS_SDK_headers_for_macOS_10.14.pkg
package, everything still works, and it's possible to use sprout-wrap
on the latest macOS & macbook models.
I've made some fixes to my bootstrap script, and finally got a successful Chef run after removing or migrating deprecated homebrew casks, recipes, etc... The related changes are in work-laptop-dependency-updates
branch if you're curious.
I've gotten sprout + soloist
working on macOS Big Sur, Monterey, and even patched the homebrew
cookbook to get Homebrew working again on the M1 / arm64 chip! (Note: as of this writing, even Chef's built-in homebrew provider does not yet support this)
In case anyone's still interested in using sprout
/ sprout-wrap
... I've managed to refactor my bootstrap script to install via rvm
ruby + gemset with Gemfile.lock
. This seems to be the way most projects are going these days, and there are some bloggers that warn against using system ruby. It seems to me that they make good points about versions changing across macOS releases, and the ensuing gem installation hassles that I'm sure we've all experienced (hence this unresolved issue).
I've moved my sprout-wrap
repository under my 'sole-proprietor' org here: LyraPhase/sprout-wrap. I've been wondering if I should create a proper hard fork & rename the project to avoid the occasional upstream accidental pull request (GitHub defaults to merging back to this repo), or if this project is going to be revived?
if I should create a proper hard fork & rename the project to avoid the occasional upstream accidental pull request (GitHub defaults to merging back to this repo), or if this project is going to be revived?
I can't speak for anyone else, but in my eyes you are the steward of sprout-wrap, and I like the idea of a proper hard fork & rename to avoid the accidental PR.
macOS (v10.12) ships with an EOL'd version of ruby (v2.0.0p648),
This is problematic for running sprout because many ruby gems are dropping support for this version of ruby. The following version locks are currently required to run sprout: https://github.com/pivotal-sprout/sprout-chruby/blob/master/Gemfile#L12-L20
There are a number of possible options for addressing the lagging ruby support in macOS.
One Idea is to replace the sprout ecosystem with something based on Brewfile plus some sort of dotfiles tooling. This would be losing the investment in cookbooks that have been created.
Another idea would be for sprout to bring its own ruby. This would increase boot-strap time but give a consistent ruby env which sprout and its associated gems could depend on.
I'm opening this issue as a place to further this discussion. Paging @wendorf, @cunnie, @hiremaga, ... Please tag anyone who might be interested.