load, fetch, etc. broken on some locales

[GoogleCodeExporter]

Because /usr/bin/rrdtools relies on the system locale to write its xml, the
output depends on the locale (and can use comas instead of dots in numbers,
for example). PyRRD wants dots to parse numbers.

PyRRD should set LC_ALL=C in the environment when calling rrdtool. This can
be done in PyRRD.external._cmd on the current release.

Currently, RRD(filename, mode='r') is broken in some locales.

Original issue reported on code.google.com by g2p.c...@gmail.com on 3 Aug 2009 at 2:14

[GoogleCodeExporter]

Thanks! I'll add a unit test that fails and then update the code so that the 
test passes.

Original comment by duncan.m...@gmail.com on 28 Oct 2009 at 12:27

[GoogleCodeExporter]

Original comment by duncan.m...@gmail.com on 29 Oct 2009 at 5:52

[GoogleCodeExporter]

Hrm... I've tried getting rrdtool to output dump data in non-US-English formats
(e.g., de_DE.UTF-8), but I've had no success in duplicating the problem. I've 
tried
doing this from Python with the locale.setlocale method, I've tried doing this 
in
_cmd with "LC_ALL=de_DE_UTF-8 rrdtool %" and I've tried directly on the command 
line.
The dump command consistently returns values in scientific notation with no 
commas,
so need an example to work from.

Can you do the following for me, so that I can write an appropriate unit test 
and get
a fix for this issue?

1) Paste an RRD create command that breaks with a non-en_US locale. This create
command should have values such that, when dumped, the commas that you 
mentioned are
present instead of periods.

2) Paste the exact locale that you're using.

I may need more, but let's see if I can duplicate the issue with that info.

Thanks!

Original comment by duncan.m...@gmail.com on 29 Oct 2009 at 9:27

• Changed state: Incomplete

[GoogleCodeExporter]

Issue 38 has been merged into this issue.

Original comment by duncan.m...@gmail.com on 18 Oct 2011 at 8:36

[GoogleCodeExporter]

For anyone who is following this issue, I can help you out better if someone 
can provide me with the info I asked for in comment #3.

Thanks!

Original comment by duncan.m...@gmail.com on 18 Oct 2011 at 8:37

[GoogleCodeExporter]

Original comment by duncan.m...@gmail.com on 18 Oct 2011 at 8:49

• Changed title: load, fetch, etc. broken on some locales

[GoogleCodeExporter]

Here follows a test script based on example3.py:

import locale

from math import sin
from random import random

from pyrrd.rrd import RRD, RRA, DS

print locale.getdefaultlocale()
print locale.getlocale()

locale.setlocale(locale.LC_ALL, '')

print locale.getdefaultlocale()
print locale.getlocale()

filename = 'example.rrd'

startTime = 1122876000
endTime = 1136012400
step = 300
maxSteps = int((endTime-startTime)/step)

# Let's create and RRD file and dump some data in it
dss = []
rras = []
ds1 = DS(dsName='speed', dsType='GAUGE', heartbeat=900)
dss.append(ds1)
rra1 = RRA(cf='AVERAGE', xff=0.5, steps=12, rows=1460)
rras.append(rra1)
myRRD = RRD(filename, ds=dss, rra=rras, start=startTime)
myRRD.create()

# let's generate some data...
currentTime = startTime
for i in xrange(maxSteps):
    currentTime += step
    # lets update the RRD/purge the buffer ever 100 entires
    if i % 100 == 0 and myRRD.values:
        #print "updating RRD..."
        myRRD.update(debug=False)
    # let's do periodic values
    value = int(sin(i % 200) * 1000)
    myRRD.bufferValue(currentTime, value + 0.1)
# add anything remaining in the buffer
myRRD.update()

print myRRD.fetch(start=startTime, end=endTime)['speed']

This produces the following output:

('it_IT', 'UTF8')
(None, None)
('it_IT', 'UTF8')
('it_IT', 'UTF8')
Traceback (most recent call last):
  File "example3.py", line 47, in <module>
    print myRRD.fetch(start=startTime, end=endTime)['speed']
  File "/usr/local/lib/python2.6/dist-packages/PyRRD-0.1.0-py2.6.egg/pyrrd/rrd.py", line 226, in fetch
    return self.backend.fetch(*data)[returnStyle]
  File "/usr/local/lib/python2.6/dist-packages/PyRRD-0.1.0-py2.6.egg/pyrrd/backend/external.py", line 143, in fetch
    data = [common.coerce(datum) for datum in data.split()]
  File "/usr/local/lib/python2.6/dist-packages/PyRRD-0.1.0-py2.6.egg/pyrrd/backend/common.py", line 29, in coerce
    raise ValueError, "Unexpected type for data (%s)" % value
ValueError: Unexpected type for data (4,6100000000e+01)

The result is the same even if I do not set the locale at the beginning.

Original comment by roberto....@gmail.com on 19 Oct 2011 at 8:45

[GoogleCodeExporter]

Roberto, 

This is a good help, thanks. I'll work with this and see what I can come up 
with for a solution.

Original comment by duncan.m...@gmail.com on 19 Oct 2011 at 2:16

• Changed state: InProgress
• Added labels: Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

Hello! Is anybody know how to make the "fetch" command for rrd file while this 
issue not solved?

Original comment by Skryab...@gmail.com on 30 Mar 2012 at 4:10