search

oetiker / rrdtool-1.x

RRDtool 1.x - Round Robin Database
http://www.rrdtool.org
GNU General Public License v2.0
1.02k stars 264 forks source link

Using the <RRD::INCLUDE> clause within an <RRD::GRAPH> clause in an rrdcgi script results in the included file not being parsed correctly. #292

Open oetiker opened 12 years ago

oetiker commented 12 years ago

Using the RRD::INCLUDE clause within an RRD::GRAPH clause in an rrdcgi script results in the included file not beingparsed correctly.

This used to work. I built a couple of stats servers in the past that relied heaily on building the rrdgraph definitions from building blocks of include files. This was 6 or 7 years ago. No matter what I do I cannot make anything included from a file using the RRD::INCLUDE clause parse correctly, no matter how trival the contents. The documentation for rrdcgi doesn't make any suggestion that this is no longer a valid way of working.

I hve tried to make this work setting DEF, CDEFs, COMMENTs and even the command line arguments to rrdgraph itself from include files. Everything works as expected outside of an RRD::GRAPH clause and produces an error if used within such a clause.

To demonstrate the issue I'm using a stripped down version of one of the rddcgi scripts from my stats server. It uses a couple of deprectated forms of things but it still works fine and generates valid data. For the purposes of this test I'm not trying to embed this in any HTML.

This is the script includetest-1.cgi {{{

!/usr/pkg/bin/rrdcgi

<RRD::GRAPH images/gateway-day-includetest-1.png
  --lazy
  -s end-1day
  -v "Bits per Second"
  -w 800 -h 250 -t "Uplink Throughput (Daily)"
  --imginfo "<IMG SRC="images/%s" WIDTH="%lu" HEIGHT="%lu">"
  DEF:rawavinoct=/home/halibut/rrd/gateway/interfaces/103/ifInOctets.rrd:data:AVERAGE
  DEF:rawavoutoct=/home/halibut/rrd/gateway/interfaces/103/ifOutOctets.rrd:data:AVERAGE
  CDEF:avinbit=0,rawavinoct,8,*,-
  CDEF:avoutbit=rawavoutoct,8,*
  CDEF:avinbitprint=0,avinbit,-
  AREA:avoutbit#0000ff:"Average out"
  GPRINT:avoutbit:AVERAGE:" %7.3lf %Sbps\n"
  AREA:avinbit#00ff00:"Average in"
  GPRINT:avinbitprint:AVERAGE:"   %7.3lf %Sbps\n"
>

}}}

Running in offline mode shows no errors. {{{ (offline mode: enter name=value pairs on standard input) Content-Type: text/html Content-Length: 74

<IMG SRC=images/gateway-day-includetest-1.png WIDTH=897 HEIGHT=337>

}}}

Now the same script again but with ian include file to add a timestamp to the graph. This is included twice. Once ioutside the RRD::GRAPH clause so that we can see that it's read correctly with in contents just ending up included in the output. It is included a second time within the RRD::GRAPH clause to try and make the timestamp work.

The script is includetest-2.cgi, a diff to includetest-1 is included. {{{ --- includetest-1.cgi 2011-01-25 13:47:11.000000000 +0000 +++ includetest-2.cgi 2011-01-25 13:50:34.000000000 +0000 @@ -1,6 +1,8 @@

!/usr/pkg/bin/rrdcgi

And this is the contents of the include file include/timestamp {{{ COMMENT:"\n" COMMENT:"Generated at <RRD::TIME::NOW '%a %b %e %H\:%M\:%S %Z %Y'>\c" }}}

And now the output, it's worth noting that RRD:: clauses within the include file are being correctly expanded, that resultant text is being inlined correctly. However, the expansion inside the RRD::GRAPH clause is generating an error. {{{ (offline mode: enter name=value pairs on standard input) Content-Type: text/html Content-Length: 190

COMMENT:"\n"

COMMENT:"Generated at Tue Jan 25 13:57:48 GMT 2011\c"

[ERROR: Could not make sense out of 'COMMENT:"\n"

COMMENT:"Generated at Tue Jan 25 13:57:48 GMT 2011\c" ']

}}}

OK. So this might be because there is something bad in those COMMENT clauses in the include file. So we'll try a couple of things. First we'll use a different include file, one as basic as it comes, an empty one.

Diff between the previous script , includetest-2.cgi and the new one including an empty file, includetest-3.cgi. {{{ --- includetest-2.cgi 2011-01-25 13:50:34.000000000 +0000 +++ includetest-3.cgi 2011-01-25 16:05:00.000000000 +0000 @@ -1,8 +1,8 @@

!/usr/pkg/bin/rrdcgi

This is the file being included. {{{ -rw-r--r-- 1 www www 0 Jan 25 13:12 include/nullfile }}}

And the output is slighty odd, it seems to be telling me there's an error in an empty string... {{{ (offline mode: enter name=value pairs on standard input) Content-Type: text/html Content-Length: 52

[ERROR: Could not make sense out of '']

}}}

We'll also try just inlining the contents of the timestamp include file just to see if it's valid.

Diff between the original script, includetest-1.cgi and this inlined version, includetest4.cgi. {{{ --- includetest-1.cgi 2011-01-25 13:47:11.000000000 +0000 +++ includetest-4.cgi 2011-01-25 16:12:35.000000000 +0000 @@ -15,5 +15,7 @@ GPRINT:avoutbit:AVERAGE:" %7.3lf %Sbps\n" AREA:avinbit#00ff00:"Average in" GPRINT:avinbitprint:AVERAGE:" %7.3lf %Sbps\n"

And the output seemed to suggest that everythign is fine with the inlined version. {{{ (offline mode: enter name=value pairs on standard input) Content-Type: text/html Content-Length: 74

<IMG SRC=images/gateway-day-includetest-1.png WIDTH=897 HEIGHT=351>

}}}

Version string from my copy of rrdtool as I don't seem to be able to get a version from rrdcgi. {{{ RRDtool 1.4.4 Copyright 1997-2010 by Tobias Oetiker tobi@oetiker.ch Compiled Sep 7 2010 15:44:42

Usage: rrdtool [options] command command_options Valid commands: create, update, updatev, graph, graphv, dump, restore, last, lastupdate, first, info, fetch, tune, resize, xport, flushcached

RRDtool is distributed under the Terms of the GNU General Public License Version 2. (www.gnu.org/copyleft/gpl.html)

For more information read the RRD manpages }}}

oetiker commented 12 years ago

[oetiker] you are correct ... there are various ways to trip rrdcgi :-( ... the only help for this would be to rewrite it from ground up providing a proper parser with recursion support ... for now it is just a sort of little hack I'm afraid ... regarding quotes, make sure to observe shell quoting rules:

--imginfo ""

kwaio commented 8 years ago

Hi, any update on this bug ? I'm trying to migrate an old mrtg server to a newer system, and it seems that cgi that used to work on rrdtool-1.0.49 are now broken on rrdtool-1.4.8 (centos 7's default version)

oetiker commented 8 years ago

well and the current version of rrdtool is 1.6.0 ... if you insist on using an old version you should report this to the maintainers of your distro

kwaio commented 8 years ago

Fair enough, I'm trying to package the latest version on my own, and will submit a request to Red Hat so they at least backport the fix.

oetiker commented 8 years ago

@kwaio please be aware that I can not promise you that the problem you are seeing is fixed in 1.6.0, but I can at least try to reproduce it and maybe there is a quick fix ... as with many opensource projects, there is only so much time we can invest for free ...

kwaio commented 8 years ago

Same here, i'll spend some time trying to make it work with a newer version, and will fallback to 1.0 if I fail after a few hours... Thanks for the (very) quick answers.