installation/running instructions

[Dieterbe]

please provide instructions on how to install this, which dependencies are needed, and how to run it. do we have to build a gem first and then install the gem? what's the command to execute the server?

[ripienaar]

It's a standard ruby rack application, installation depends on many many factors like what webserver you use - but its installed just like any other sinatra or rack application so its not needed to document that here when there are 100s of other good docs about this already.

The specific stuff that needs to be done to configure the application once deployed like any other rack app is documented here.

We publish a gdash gem to rubygems already.

[dillera]

Try this:

http://yehudakatz.com/2010/04/02/using-gemspecs-as-intended/

I had to add some lines to the top of the gdash.gemspec to get it to work. Put these lines at the top of the gdash.gemspec_

lib = File.expand_path('../lib/', __FILE__)
$:.unshift lib unless $:.include?(lib)

require 'rake'

Then you just build it and install it:

[gdash ]$ vi gdash.gemspec 
[gdash ]$ gem build gdash.gemspec 
  Successfully built RubyGem
  Name: gdash
  Version: 0.0.4
  File: gdash-0.0.4.gem
[gdash ]$ gem install gdash-0.0.4.gem
Successfully installed gdash-0.0.4
1 gem installed
[gdash ]$ 

And then I run it like this:

nohup rackup config.ru &> /dev/null &

And you are good to go.

[Dieterbe]

@ripienaar IMHO you shouldn't expect of your users they should research and familiarize themselves with the entire build and installation process of ruby apps (if they don't want to). isn't this application targeted at people who just want to run a dashboard for graphite? I think it would make sense to just document the needed steps to get it up and running, so that those of us who are not familiar with ruby can get it running quickly. If it wasn't for @dillera (thanks btw!) it would have taken me much time to have figured out those missing lines in the gemspec, because without these lines it didn't work indeed. i got this error:

dieter.plaetinck@dfvimeographite1:~/gdash$ gem build gdash.gemspec
Invalid gemspec in [gdash.gemspec]: uninitialized constant Gem::Specification::FileList
ERROR:  Error loading gemspec. Aborting.

and do I have to install the gem as root? I'm getting this:

dieter.plaetinck@dfvimeographite1:~/gdash$ gem install gdash-0.0.2.gem 
Fetching: graphite_graph-0.0.4.gem (100%)
ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions into the /usr/lib/ruby/gems/1.8 directory.

(notice also how the gem that is built is 0.0.2 but it tries to install 0.0.4)

[ripienaar]

you dont need to build the gem, its on rubygems, just gem install gdash and it will pull in whatever dependencies it needs.

Like I pointed out the install process varies massively between webservers, local deploy methods, do you use bundler or not. It's a problem with the ruby ecosystem, too many options and no real single direction.

I cannot document every possible install process.

The general ruby trend now is to use bundler, the gem is setup for bundler. But as you can see from above examples not everyone does it this way - some dont use bundler as there are many methods some people even think they need to edit libdirs and whatever but its not needed. How you do it will also differ based on what you want to run the app in, I use apache and passenger for my production deploy othrs use nginx, or unicorn or just webrick, many many ways.

So for me I chose the bundler method so install bundler with "gem install bundler".

So I create a new empty directory where i want to isntall it and set it up like:

% ls
config  config.ru  Gemfile graph_templates

the config is as per the readme as is graph_templates, config.ru is the one in the top of the github repo.

The Gemfile is here:

source 'http://rubygems.org'
gem 'gdash'

Now all you have to do to install gdash, the graph dsl and every dependency it needs:

% bundle install --deployment  
Installing graphite_graph (0.0.4) 
Using redcarpet (2.1.1) 
Using rack (1.4.1) 
Using rack-protection (1.2.0) 
Using tilt (1.3.3) 
Using sinatra (1.3.2) 
Installing gdash (0.0.4) 
Using bundler (1.1.3) 
Updating .gem files in vendor/cache
Your bundle is complete! It was installed into ./vendor/bundle

Note this can be run as a normal user and it installs into vendor all the dependnecies, no need to use root or modify the system in any way

Run it:

% bundle exec rackup  
[2012-07-12 19:12:57] INFO  WEBrick 1.3.1
[2012-07-12 19:12:57] INFO  ruby 1.8.7 (2011-06-30) [x86_64-linux]
[2012-07-12 19:12:57] INFO  WEBrick::HTTPServer#start: pid=12297 port=9292

[dillera]

Its very useful to be able to edit some things locally in the source, then build your own gem and use it. This is what I need to do, and pulling the gem from rubygems is not a good solution in all cases.

[ripienaar]

@dillera yeah, not against it and wont do anything to stop it - but it just shows further that everyone has their way, no generic install method exist

[dillera]

@Dieterbe -

You should definitely use rbenv: (and then you won't have to use root)

https://gist.github.com/2760597

This will allow you to keep multiple Rubies in your home dir/user space and leave the system ruby alone.

Also, you are building 0.0.4 if you clone this and built it. Perhaps 0.0.2 is on rubygems?

[alrighttheresham]

Guys I have no experience in Ruby, can I ask that if anyone has a configuration that works on Centos can they share the instructions please.

[ninowalker]

@alrighttheresham This is a fair criticism. @ripienaar A simple paragraph in the README on how to build/install would be helpful. Not being a ruby-ite I'm of little use here :/

[dillera]

He gave a fine example above, the most generic install is one command:

gem install gdash

If you are asking for directions on how to use it, that is another thing.

[ripienaar]

I also explained why it's not possible - there are literally 10s of ways to deploy a ruby application. It varies massively from distro to distro, rvm to rbenv to chruby and every other crazy thing people do with ruby.

It's just not possible - however this application complies with the design of any other Rack application so any guide about Rack and your distribution should apply just fine.

[alrighttheresham]

ripienaar I understand your position, problem is there is nowhere that shows how to quickly put all this together with even one use case / combination. Since I commented on this, I have a found a solution that works on centos 6.3. For people like me that dont understand the nuances of ruby please see below.

Hopefully this is helpful to others, I recognise that there are things here that may be done better, but it works and allows me to continue evaluating gdash as a solution. The passenger module part of the apache config is brittle, but other than that its probably ok.

Again ripienaar thanks for the product, so far it looks like its a great fit for our needs.

Graphite dependencies

      yum -y install pycairo mod_python Django django-tagging python-ldap python-memcached python-sqlite2 python-twisted bitmap bitmap-fonts python-devel python-crypto pyOpenSSL httpd mod_wsgi python-zope-interface memcached

      # Install Node
      NODE_VERSION=v0.8.15
      wget http://nodejs.org/dist/$NODE_VERSION/node-$NODE_VERSION.tar.gz
      tar xvf node-$NODE_VERSION.tar.gz
      pushd node-$NODE_VERSION
      ./configure
      make install
      popd
      rm -rf node-$NODE_VERSION.tar.gz

      # Install statsd 
      pushd /opt
      git clone git://github.com/etsy/statsd.git
      popd

      # Download and Install Graphite
      GRAPHITE_BASE=0.9
      GRAPHITE_VERSION=0.9.9
      wget https://launchpad.net/graphite/$GRAPHITE_BASE/$GRAPHITE_VERSION/+download/whisper-$GRAPHITE_VERSION.tar.gz
      wget https://launchpad.net/graphite/$GRAPHITE_BASE/$GRAPHITE_VERSION/+download/carbon-$GRAPHITE_VERSION.tar.gz
      wget https://launchpad.net/graphite/$GRAPHITE_BASE/$GRAPHITE_VERSION/+download/graphite-web-$GRAPHITE_VERSION.tar.gz

      tar xvf whisper-$GRAPHITE_VERSION.tar.gz
      pushd whisper-$GRAPHITE_VERSION
      sudo python setup.py install
      popd
      rm -rf whisper-$GRAPHITE_VERSION

      tar xvf carbon-$GRAPHITE_VERSION.tar.gz
      pushd carbon-$GRAPHITE_VERSION
      sudo python setup.py install
      popd
      rm -rf carbon-$GRAPHITE_VERSION

      tar xvf graphite-web-$GRAPHITE_VERSION.tar.gz
      pushd graphite-web-$GRAPHITE_VERSION
      sudo python setup.py install
      popd
      rm -rf graphite-web-$GRAPHITE_VERSION

      # configure apache
      cat > /etc/httpd/conf.d/graphite.conf <<EOF
       Listen 8080
      <VirtualHost *:8080>
        DocumentRoot "/opt/graphite/webapp"
        ErrorLog logs/graphite_error_log
        TransferLog logs/graphite_access_log
        LogLevel warn
        WSGIDaemonProcess graphite processes=5 threads=5 display-name=" {GROUP}" inactivity-timeout=120
        WSGIProcessGroup graphite
        WSGIScriptAlias / /opt/graphite/conf/graphite.wsgi
        Alias /content/ /opt/graphite/webapp/content/
        <Location "/content/">
         SetHandler None
        </Location>
        Alias /media/ "/usr/lib/python2.6/site-packages/django/contrib/admin/media/"
        <Location "/media/">
         SetHandler None
        </Location>
        <Directory /opt/graphite/conf/>
         Order deny,allow
         Allow from all
        </Directory>
      </VirtualHost>
      EOF

      echo "WSGISocketPrefix /var/run/wsgi" >> /etc/httpd/conf.d/wsgi.conf 

      # configure graphite
      pushd /opt/graphite/conf
      cp carbon.conf.example carbon.conf
      cp storage-schemas.conf.example storage-schemas.conf
      cp graphite.wsgi.example graphite.wsgi
      popd

      pushd /opt/graphite/webapp/graphite/
      python manage.py syncdb --noinput
      popd

      pushd /opt/graphite/webapp/graphite
      cp local_settings.py.example local_settings.py
      popd

      # Need to setup permissions on the storage directory so apache can write to it
      chown -R apache:apache /opt/graphite/storage/

      cat > /etc/init.d/carbon-cache <<EOF
      #!/bin/bash
      #
      # This is used to start/stop the carbon-cache daemon

      # chkconfig: 345 90 10
      # description: Starts the carbon-cache daemon

      # Source function library.
      . /etc/init.d/functions

      RETVAL=0
      prog="carbon-cache"

      start_relay () {
          /usr/bin/python /opt/graphite/bin/carbon-relay.py start
              RETVAL=\$?
              [ \$RETVAL -eq 0 ] && success || failure
              echo
              return \$RETVAL
      }

      start_cache () {
           /usr/bin/python /opt/graphite/bin/carbon-cache.py start
              RETVAL=\$?
              [ \$RETVAL -eq 0 ] && success || failure
              echo
              return \$RETVAL
      }

      stop_relay () {
          /usr/bin/python /opt/graphite/bin/carbon-relay.py stop
              RETVAL=\$?
              [ \$RETVAL -eq 0 ] && success || failure
              echo
              return \$RETVAL
      }

      stop_cache () {
                /usr/bin/python /opt/graphite/bin/carbon-cache.py stop
              RETVAL=\$?
              [ \$RETVAL -eq 0 ] && success || failure
              echo
              return \$RETVAL
      }

      # See how we were called.
      case "\$1" in
        start)
          #start_relay
          start_cache
              ;;
        stop)
          #stop_relay
          stop_cache
              ;;
        restart)
          #stop_relay
          stop_cache
          #start_relay
          start_cache
          ;;

        *)
              echo \$"Usage: \$0 {start|stop}"
              exit 2
              ;;
      esac
      EOF
      chmod 0755 /etc/init.d/carbon-cache

      # configure statsd
      cat > /opt/statsd/local.js <<EOF
      /*

      Required Variables:

        port:             StatsD listening port [default: 8125]

      Graphite Required Variables:

      (Leave these unset to avoid sending stats to Graphite.
       Set debug flag and leave these unset to run in 'dry' debug mode -
       useful for testing statsd clients without a Graphite server.)

        graphiteHost:     hostname or IP of Graphite server
        graphitePort:     port of Graphite server

      */
      {
        graphitePort: 2003
      , graphiteHost: "33.33.33.10"
      , port: 8125
      , address: "33.33.33.10"
      }
      EOF

      cat > /etc/init.d/statsd <<EOF
      #!/bin/bash
      #
      # StatsD
      #
      # chkconfig: 345 99 01
      # description: StatsD init.d
      #
      . /etc/rc.d/init.d/functions

      lockfile=/var/lock/subsys/statsd
      RETVAL=0
      STOP_TIMEOUT=\${STOP_TIMEOUT-10}

      start() {
        echo -n \$"Starting statsd: "
        cd /opt/statsd

        # See if it's already running. Look *only* at the pid file.
        if [ -f /var/run/statsd.pid ]; then
            failure "PID file exists for statsd"
            RETVAL=1
        else
            # Run as process
            /usr/local/bin/node ./stats.js /opt/statsd/local.js >> /var/log/statsd.log 2>> /var/log/statsderr.log &
            RETVAL=\$?

            # Store PID
            echo \$! > /var/run/statsd.pid

            # Success
            [ \$RETVAL = 0 ] && success "statsd started"
        fi

        echo
        return \$RETVAL
      }

      stop() {
        echo -n \$"Stopping statsd: "
        killproc -p /var/run/statsd.pid
        RETVAL=\$?
        echo
        [ \$RETVAL = 0 ] && rm -f /var/run/statsd.pid
      }

      # See how we were called.
      case "\$1" in
        start)
        start
        ;;
        stop)
        stop
        ;;
        status)
        status -p /var/run/statsd.pid \${prog}
        RETVAL=\$?
        ;;
        restart)
        stop
        start
        ;;
        condrestart)
        if [ -f /var/run/statsd.pid ] ; then
            stop
            start
        fi
        ;;
        *)
        echo \$"Usage: statsd {start|stop|restart|condrestart|status}"
        exit 1
      esac

      exit \$RETVAL
      EOF
      chmod 0755 /etc/init.d/statsd

      yum -y install curl-devel httpd-devel apr-devel apr-util-devel
      gem install rdoc
      gem install bundler
      gem install passenger
      passenger-install-apache2-module -a
      cat >> /etc/httpd/conf.d/passenger.conf  <<EOF
         LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-3.0.18/ext/apache2/mod_passenger.so
         PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-3.0.18
         PassengerRuby /usr/bin/ruby

         <VirtualHost *:80>
         RackAutoDetect On
         DocumentRoot /opt/gdash/public
         ErrorLog   /var/log/httpd/gdash_error_log
         CustomLog   /var/log/httpd/gdash_access_log combined
         <Directory /opt/gdash/>
            # This relaxes Apache security settings.
            AllowOverride all
            # MultiViews must be turned off.
            Options -MultiViews
            allow from all
         </Directory>
         </VirtualHost>
      EOF
      pushd /opt
      git clone https://github.com/ripienaar/gdash.git
      pushd /opt/gdash
      bundle install
      popd
      cp /opt/gdash/config/gdash.yaml-sample /opt/gdash/config/gdash.yaml
      sed -i 's/\/path\/to\/my\/graph\/templates/\/opt\/gdash\/graph_templates\/dashboards/g' /opt/gdash/config/gdash.yaml
      sed -i 's/graphite.example.net/33.33.33.10:8080/g' /opt/gdash/config/gdash.yaml
      sed -i 's/\/var\/lib\/carbon\/whisper/\/opt\/graphite\/storage\/whisper\//g' /opt/gdash/config/gdash.yaml
      chown -R apache:apache /opt/gdash
      popd 

      # Start services on boot
      chkconfig httpd on 
      chkconfig --level 345 carbon-cache on 
      chkconfig --level 345 statsd on

[ripienaar]

@alrighttheresham do you want to contribute a README.redhat.md or something similar into the project? I'd be glad to merge something like that.

Or if you have a blog - blog about it and we'll link to that.

However, you're example shows how a "A simple paragraph in the README" just won't cut it :)

[alrighttheresham]

@ripienaar no blog, too old ;-) I put what I did recently on github, which was a Vargrant based CentOS VM with gdash / statsd / graphite on github. Link is available here.

Feel free to use / copy / promote any way you feel appropriate.

[ripienaar]

would be great if you send a pull request with your work in it - then you'll get the right credit etc

[ninowalker]

This is great stuff guys. I was thinking a paragraph just for gdash :). e.g:

cd gdash/
sudo gem update # go read War and Peace
sudo gem install bundler
sudo gem install foreman
bundle install --deployment

cp config/gdash.yaml-sample config/gdash.yaml
# edit config/gdash.yaml
# add some graphs
# create a Procfile: http://ddollar.github.com/foreman/

foreman start

Or something to that effect

[sodabrew]

No, do not recommend 'sudo gem update'.

[ninowalker]

@sodabrew - understood. As this is an operations-centric tool, whereby you really don't need to be a rubyist to desire to use it, or even need to modify it, a yellow-brick road of instructions based on current ruby best practices would go a long way to making it broadly accessible.

[sodabrew]

Ok, you don't understand. As part of setting up one application, you would update gems in the local bundle. Not the entire system. Period.

[ninowalker]

Indeed, that seems to be true. Thanks for pointing that out.

Your point reenforces my earlier suggestion: advise a handful commands to run that are "ruby best practices", that, if your system isn't corrupted like mine is now, should get you up in no time…

Graphite, while being python project, doesn't require that you know python practices; instead they give you a path to get it running (e.g. http://graphite.wikidot.com/installation). Whereas, an experienced python developer will know how/what to modify in the process if they have special needs. That's all I'm asking :)

On Thu, Dec 13, 2012 at 10:20 AM, Aaron Stone notifications@github.comwrote:

Ok, you don't understand. As part of setting up one application, you would update gems in the local bundle. Not the entire system. Period.

— Reply to this email directly or view it on GitHubhttps://github.com/ripienaar/gdash/issues/45#issuecomment-11346104.

[sodabrew]

I totally agree that there should be a "here's one simple way of getting this running, see also { other links } for larger installations" README-style doc, +1 to that. Sorry for the strong response, yes it can be a steep learning curve to deploy a Ruby app.