search

railsmachine / moonshine

Simple Rails deployment and configuration management. 15 minute deploys of Rails 2, 3, or 4 apps.
http://railsmachine.github.com/moonshine/
GNU Lesser General Public License v3.0
666 stars 70 forks source link

Automated ruby wrapper to optimize GC variables #122

Closed rylwin closed 12 years ago

rylwin commented 12 years ago

I've just spent a while messing around with creating a wrapper for ruby to set GC variables as described in this blog post (http://www.coffeepowered.net/2009/06/13/fine-tuning-your-garbage-collector/).

I'd like to automate the modification of passenger.conf and the creation of the ruby wrapper script. Is this something that I should write as a moonshine plugin or is this useful (common) enough to write into moonshine itself?

technicalpickles commented 12 years ago

Good question. I could see it going either way.

One thought that occurred to me is that it'd almost make sense to have those always be defined in the environment, so you don't have to specifically recall to use this wrapper script. Also, less changing of the passenger configuration.

What do you think?

rylwin commented 12 years ago

I like the idea of defining the GC settings in the environment, but it turns out that this really isn't achievable. After some trial and error I started searching and came across this post: http://blog.phusion.nl/2008/12/16/passing-environment-variables-to-ruby-from-phusion-passenger/

Given the wrapper seems necessary there are two "features" that moonshine would need to support to automate this:

  1. Make PassengerRuby in passenger.conf.erb a configurable variable (default can still be /usr/bin/ruby).
  2. Add a recipe to create the wrapper script with settings that could be provided in moonshine.yml or the app manifest

It seems like item 1 would be best to change in moonshine itself, and it would be backwards compatible. Item 2 I'm not sure if it's better off as a plugin or part of moonshine, I guess it depends on how many people would want to customize these GC settings. I see it as critical given the huge performance boost I received by tweaking the GC settings, but perhaps that's just me.

rylwin commented 12 years ago

Perhaps I'm making this too complex. Maybe the only thing that would have to change in moonshine is item 1 from my previous post: make PassengerRuby path configurable. Then you could just make the wrapper script and put in in ./lib and set PassengerRuby to point to /home/user/deploy_path/lib/ruby-with-env

And making PassengerRuby configurable can also be done by moving passenger.conf.erb to app/manifests/templates.

So now I have myself wondering, is it worth making a change to moonshine to make all of this easier?

technicalpickles commented 12 years ago

It's a pretty easy thing to make configurable, so I don't think there's anything to lose.

However, it's worth thinking about how you would be tuning this thing. It's easy enough to expose configuring the GC variables, but the real challenge is if there's a reasonable set of defaults, and if not, how can you lead people to it? For mysql, for example, mysqltuner.pl can lead you to enlightenment. Not sure if there's something equivalent.

rylwin commented 12 years ago

I think it's possible to select defaults that, though not optimized for your specific rails app, would yield better performance than the ruby defaults. For example, just copying/pasting the config given in the link in the OP led to ruby creating 3 heaps on startup instead of the 9 it does by default--big performance boost. I then tweaked further and was able to get down to creating a single heap.

I got the relevant data to tune using https://github.com/mogest/scrap, which gives you GC stats by going to http://yoururl.com/stats/scrap. I suppose I could pull out the relevant stats provided by scrap into a rake task that could load the environment to get to the same GC data (since seeing the heap stats on startup seems to be enough to tweak the settings) and provide a recommendation based on that.

I would think the recommendation would be based on getting down to allocating a single heap on startup with some additional buffer (~10%?). To do this, one just needs to count the number of objects allocated when the Rails app starts and add a bit of buffer to that number--that would handle what's probably the most important GC setting and could be provided by the rake task.

The other settings are either based off of this number or would involve tracking long-term app performance. For the latter it may be better to decide upon some reasonable defaults (e.g., there seems to be a consensus that RUBY_HEAP_SLOTS_GROWTH_FACTOR should be 1 for a Rails app).

technicalpickles commented 12 years ago

Sorry for the belated update, but support for setting these via apache's envvars was added back in 20e16c7042e4629e0aa0f229164dbc0323d74c88.

I would still like to figure something out for defaults eventually, or at least some automated/methodical way to do tuning. I think we got enough to close this out for now though :)

rylwin commented 12 years ago

Awesome. Thanks for letting me know!

When I get the time I'll play around with setting the envvars via apache to make sure it works (for some reason I feel like I tried this once and didn't get it to work...but maybe I wasn't setting it right).

Automatic tuning would be cool. I guess the general concept would be to have a script that would launch a server instance (preferably in the production mode) and make a request to see how many objects were created to get the app server up and running. Then this number could be used to back into some reasonable settings.