Can't use the bundle install task with a read only installation of Ruby

[iphoting]

Hi,

I have system-wide RVM setup and the bamboo user is part of the RVM group.

java.lang.IllegalArgumentException: Can't use the bundle install task with a read only installation of Ruby.
    at au.id.wolfe.bamboo.ruby.tasks.bundler.BundlerTask.buildCommandList(BundlerTask.java:28)
    at au.id.wolfe.bamboo.ruby.tasks.BaseRubyTask.execute(BaseRubyTask.java:53)
    at com.atlassian.bamboo.task.TaskExecutorImpl.executeTasks(TaskExecutorImpl.java:183)
<snip>

Is there anyway to allow the bundle install task alter my ruby gemset? Otherwise, it's impossible to activate gems that are bleeding-edge on git.

[wolfeidau]

Yeah I planned to remove this restriction when I had a chance to test how it would work. I wasn't aware you could just put the bamboo user in a group to enable installation of gems via RVM.

I had not seen the issue with git based gem installs, never even considered this!

I will cut a new release tonight to disable it and test it, this should be easier now that I know how to break it using git based gem dependencies..

[iphoting]

Thanks for the quick response! Let me know how it goes. =)

[iphoting]

Similarly, it'll be cool if it is possible to have the bundler task accept arguments.

This way, I can use run bundle config under the selected RVM environment (for problematic gems that need flags to compile), and also to feed arguments to bundle install such as --without development and/or --path .bundle/vendor (to workaround the gem install-rights issue).

[wolfeidau]

After doing the testing of bundler it seems pretty damn hell bent on using SUDO with the RVM system install.

To do so in my case I added the following to sudoers file.

    # This permits running of SUDO without a TTY
    Defaults:tomcat6 !requiretty

    # This enables it to cp / mv stuff from it's stage directory to gems directory.
    # Need to look at constraining this a bit to a specific path ideally.
    %rvm ALL=NOPASSWD: ALL

And yes I hade to create .bundle directory in tomcat6 users home directory and give it permission to write to that path.

    mkdir /usr/share/tomcat6/.bundler
    chown tomcat6:tomcat6 /usr/share/tomcat6/.bundler

With argument at the moment I have been a bit cautious only permitting single attributes or flagged to be passed, this is primarily to dodge escaping stuff.

Will need to test that a bit more as I think it will require a little thought.

Going to push my changes, do a release and update the README.

[wolfeidau]

By the way I am having a good read over the bundle help on the sub commands to get an idea of how they work.

I am not sure what you think but I would rather a build look like this.

• Checkout Code
• BundleConfig tweak setting a
• BundleConfig tweak setting b
• BundleConfig tweak setting c
• BundleInstall
• Rake db migrate
• Rake db seed
• Rake spec

Rather than one Bundle entry with all the tweaks in a string passed in, but I am certainly open to suggestions.

[iphoting]

Yes, the build process that you've suggested is perfect.

So we basically need a bundle task that allows us to specify (via textbox) the subcommands and arguments to feed it. This way, you don't need to create a separate "module or plugin" for bundle config and bundle install, just a bundle <subcommand> would suffice.

The plan would look like this:

• checkout task.
• bundle task: config <arguments to tweak setting a>
• bundle task: config <arguments to tweak setting b>
• bundle task: install --path .bundle/vendor --without development
• rake task: db:migrate
• rake task: spec

---

As for system-wide RVM, it's strange. If RVM is installed correctly and the the startup script is sourced appropriately, there's no need for sudo. I've got system-wide RVM installed on 3 machines and once the user is in the rvm group (remember to logout and login for it to take effect), just a gem install <name> works without sudo, and it won't prompt for password.

Alternatively, --path .bundle/vendor will rid the need for permissions tweaking altogether as it installs it within the working directory.

[wolfeidau]

Firstly I will get hacking on the changes to bundler task over the next few days, your suggested layout sounds great I just need to add a bit more logic to my plugin to enable it.

---

Secondly if you have a read over the bundler install documentation at the following URL.

http://gembundler.com/man/bundle-install.1.html

You will note the following statement:

In some cases, that location may not be writable by your Unix user. In that case, bundler will stage everything in a temporary directory, then ask you for your sudo password in order to copy the gems into their system location.

Will have a look in the sources to see how bundler decides whether a folder is writable, at the moment it seems to be flawed.

[iphoting]

Here are the env vars that my system-wide RVM exports in one of my projects:

export PATH ; PATH="/usr/local/rvm/gems/ruby-1.9.2-p320@octopress/bin:/usr/local/rvm/gems/ruby-1.9.2-p320@global/bin:/usr/local/rvm/rubies/ruby-1.9.2-p320/bin:/usr/local/rvm/bin:$PATH"
export rvm_env_string ; rvm_env_string='ruby-1.9.2-p320@octopress'
export rvm_path ; rvm_path='/usr/local/rvm'
export rvm_ruby_string ; rvm_ruby_string='ruby-1.9.2-p320'
export rvm_gemset_name ; rvm_gemset_name='octopress'
export RUBY_VERSION ; RUBY_VERSION='ruby-1.9.2-p320'
export GEM_HOME ; GEM_HOME='/usr/local/rvm/gems/ruby-1.9.2-p320@octopress'
export GEM_PATH ; GEM_PATH='/usr/local/rvm/gems/ruby-1.9.2-p320@octopress:/usr/local/rvm/gems/ruby-1.9.2-p320@global'
export MY_RUBY_HOME ; MY_RUBY_HOME='/usr/local/rvm/rubies/ruby-1.9.2-p320'
export IRBRC ; IRBRC='/usr/local/rvm/rubies/ruby-1.9.2-p320/.irbrc'
unset MAGLEV_HOME
unset RBXOPT

Looking at the logs, the same env vars were exported. it doesn't make sense why sudo is required in this case.

Currently, to work around the read-only issue, I've been manually executing bundle install (from RVM path) via a script and I don't see it prompt for a password. Maybe the tomcat user hasn't been restarted since it was added to the RVM group?

But still, i think this is a very small issue as it can be easily worked around by feeding bundle install with --path <location> to install gems to <location> within the working directory.

[wolfeidau]

Thanks for posting this information it is very insightful and will be great for later reference.

Currently poking around in bundler sources trying to figure why it is using sudo, I have some clues just need to prove my theory.

Adding this switch to to the Bundle Task in my plugin is quite easy. In a couple of days I will do a release with the following new options to the Bundle Task.

• --path with a box to enter one.
• --bin-stubs checkbox to enable this.

[iphoting]

That's great! Thanks for looking into it! =)

Oh yeah just FYI, the following command can be used to generate the env vars from RVM:

rvm env . -- --env

[wolfeidau]

So I had a read over code to bundler on the weekend and did some testing.

Turns out the way I had things configured bundler didn't have write access, once corrected no more sudo.

Now after some reading I deduced that the --path will need to be passed to all subsequent calls to bundler.

bundle --path /tmp/bob install
bundle --path /tmp/bob exec rake db:migrate

In my current plugin each Task is configured independently.

In the future i need to have "common" configuration, which is configured once for the build and attributes configured for each task.

I will do a bit more testing today.

[iphoting]

Hmm, that's strange. I'm under the impression that bundler will create a file under .bundle/config to store any configuration or flags used.

[wolfeidau]

OK when I was testing this I was triggering a number of builds at once, which may have caused issues with this method.

Currently I think it uses $HOME/.config as it's reference location, it would be ideal if this was actually the "work" area for the staged sources you where building.

I will dig deeper, either way I will upload a snapshot you can test out if your willing to help.

[iphoting]

I'd love to help test. Let me know how I can install the snapshot.

[wolfeidau]

Ok been wrestling with the best way to do this and have finally settled on and blogged about how I plan to do this.

http://www.wolfe.id.au/2012/06/11/bundler-gems-and-binstubs/

[iphoting]

Looks good!

[wolfeidau]

Finally had a chance to test if my latest build solves this issue, from what i can see it does.

I have just uploaded a new release for testing

http://repo.wolfe.id.au/au/id/wolfe/bamboo/rake-bamboo-plugin/1.8/

Download the rake-bamboo-plugin-1.8.jar from their if you want to test it out.

Ensure you configure the bundle task and enter the path as vendor/bundle and tick binstubs.

The make sure the rake tasks are run with bundle exec.

[wolfeidau]

Note you still need to have bundler installed in this gemset.

Mine was as follows.

gem list

*** LOCAL GEMS ***

bundler (1.1.3)
rubygems-bundler (0.2.8)

[iphoting]

Ah, works well here!

BTW, is it possible to have bundler config task in addition to the current bundler install task? (Reference: https://github.com/wolfeidau/rake-bamboo-plugin/issues/18#issuecomment-5441271).

[wolfeidau]

Interested to know what you would like to configure?

BTW now that bundler is running correctly the .bundle folder is in the working directory where it should be.

/var/atlassian/bamboo/xml-data/build-dir/RAILS-CORE-JOB1# cat .bundle/config 
---
BUNDLE_PATH: vendor/bundle
BUNDLE_BIN: bin
BUNDLE_DISABLE_SHARED_GEMS: '1'

Big win.

[iphoting]

Great job on the .bundle/config!

As for bundle config, it'll come in handy when one or two gems require some CFLAGS or the like before they will compile cleanly. You won't see it all the time, but when it happens, this option would be fantastic.

Otherwise, I think manually cat the .bundle/config file with the required flags within a script before the bundler task might solve it too.

[wolfeidau]

I will add a new issue for bundle config, going to resolve this one :)