Updates when deleting lines from global attribute files not written

[dvalters]

You can append lines to an attribute file, but trying to remove these extra lines appears to have no effect on the netcdf file. (The write operation appears to complete successfully but re-opening the attribute files shows it has not had the desired effect (The old lines are still there)

From #36

[blazk]

I just had this problem as well :-O

[dvalters]

It's the same for editing dimension variables too:

Mount a file and edit DATA_REPR for latitude in vim (for example)

Change the first line from 90.0 to 89.0 save and exit vim (Nothing suspicious reported in debug mode?)

ncdump FILE -v latitude:

 latitude = 90, 87.5, 85, 82.5, 80, 77.5, 75, 72.5, 70, 67.5, 65, 62.5, 60,   <-- no change!
    57.5, 55, 52.5, 50, 47.5, 45, 42.5, 40, 37.5, 35, 32.5, 30, 27.5, 25, 
    22.5, 20, 17.5, 15, 12.5, 10, 7.5, 5, 2.5, 0, -2.5, -5, -7.5, -10, -12.5, 
    -15, -17.5, -20, -22.5, -25, -27.5, -30, -32.5, -35, -37.5, -40, -42.5, 
    -45, -47.5, -50, -52.5, -55, -57.5, -60, -62.5, -65, -67.5, -70, -72.5, 
    -75, -77.5, -80, -82.5, -85, -87.5, -90 ;
}

[blazk]

I guess it's because "truncate" function is not implemented (does nothing). I'm trying to implement truncate code for attributes now to see if it fixes the issue

[blazk]

no,sorry I think it is different from what you describe, when I use Vim to edit attribute by deleting a line and save and open again, the contents of the attribute are different, but there is garbage at the end of attribute (like if file was not truncated)

[blazk]

Before my last pull request I could not save my Vim edits to attribute at all because "truncate()" call was failing. Maybe try merging master into your variable editing branch and see if that helps with what you describe ?

[dvalters]

No I merged the branches into master (#38) but doesn't seem to fix it - also there is traceback error from the truncate call now when editing a file:

open(u'/latitude/DATA_REPR', 32769)
getxattr(u'/latitude/DATA_REPR', u'security.capability')
getxattr(u'/latitude/DATA_REPR', u'security.capability')
removexattr(u'/latitude/DATA_REPR', u'security.capability')
truncate(u'/latitude/DATA_REPR', 0, 0L)
Uncaught exception from FUSE operation ftruncate, returning errno.EINVAL.
Traceback (most recent call last):
  File "/home/dav/devel/fusepy/fuse.py", line 715, in _wrapper
    return func(*args, **kwargs) or 0
  File "/home/dav/devel/fusepy/fuse.py", line 995, in ftruncate
    length, fh)
  File "/home/dav/devel/fusepy/fuse.py", line 1057, in __call__
    return getattr(self, op)(*args)
  File "../fusenetcdf/fusenetcdf.py", line 662, in newfunc
    result = attr(*args, **kwargs)
TypeError: truncate() takes exactly 3 arguments (4 given)
getxattr(u'/latitude/DATA_REPR', u'security.capability')
getxattr(u'/latitude/DATA_REPR', u'security.capability')
removexattr(u'/latitude/DATA_REPR', u'security.capability')
getattr(u'/latitude/DATA_REPR', None)
write(u'/latitude/DATA_REPR', '89.000000\n87.5

(in debug mode)

[blazk]

@dvalters I just pulled your current master and tried to edit DATA_REPR of latitude, I cannot write the file because I get the error:

Checking if global attr: latitude
Path is a DIMENSION VARIABLE: /latitude/DATA_REPR
Write buffer is of type: <type 'str'>
Converting write buffer to numpy array.
New array to append to VARIABLE /latitude/DATA_REPR is of TYPE: <type 'numpy.ndarray'>,Numpy DTYPE: float64
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/fuse.py", line 495, in _wrapper
    return func(*args, **kwargs) or 0
  File "/usr/local/lib/python2.7/dist-packages/fuse.py", line 604, in write
    offset, fh)
  File "/usr/local/lib/python2.7/dist-packages/fuse.py", line 800, in __call__
    return getattr(self, op)(*args)
  File "fusenetcdf/fusenetcdf.py", line 662, in newfunc
    result = attr(*args, **kwargs)
  File "fusenetcdf/fusenetcdf.py", line 725, in write
    return self.ncfs.write(path, buf, offset, fh)
  File "fusenetcdf/fusenetcdf.py", line 600, in write
    dimvar[offset:offset+len(buf)] = newdimvar
  File "netCDF4/_netCDF4.pyx", line 3996, in netCDF4._netCDF4.Variable.__setitem__ (netCDF4/_netCDF4.c:43651)
  File "netCDF4/_netCDF4.pyx", line 4208, in netCDF4._netCDF4.Variable._put (netCDF4/_netCDF4.c:45531)
RuntimeError: NetCDF: Operation not allowed in define mode

[dvalters]

Oh interesting - I had that error a while ago, but closed this issue because I could not reproduce it again:

Issue: https://github.com/dvalters/fuse-netcdf/issues/28

(I don't get that error on my current master branch)

[blazk]

@dvalters do you get any "truncate" traceback error when editing attribute?

[dvalters]

Yes I get the same:

This for a variable's attribute:

open(u'/blh/newattr', 32769)
getxattr(u'/blh/newattr', u'security.capability')
getxattr(u'/blh/newattr', u'security.capability')
removexattr(u'/blh/newattr', u'security.capability')
truncate(u'/blh/newattr', 0, 0L)
Uncaught exception from FUSE operation ftruncate, returning errno.EINVAL.
Traceback (most recent call last):
  File "/home/dav/devel/fusepy/fuse.py", line 715, in _wrapper
    return func(*args, **kwargs) or 0
  File "/home/dav/devel/fusepy/fuse.py", line 995, in ftruncate
    length, fh)
  File "/home/dav/devel/fusepy/fuse.py", line 1057, in __call__
    return getattr(self, op)(*args)
  File "../fusenetcdf/fusenetcdf.py", line 662, in newfunc
    result = attr(*args, **kwargs)
TypeError: truncate() takes exactly 3 arguments (4 given)
getxattr(u'/blh/newattr', u'security.capability')
getxattr(u'/blh/newattr', u'security.capability')
removexattr(u'/blh/newattr', u'security.capability')
getattr(u'/blh/newattr', None)
Checking if global attr: blh
write(u'/blh/newattr', 'foobazbar\n', 0, 0L)

[blazk]

@dvalters I don't understand why your code is calling truncate with three parameters

...
truncate(u'/blh/newattr', 0, 0L)
...

In my debug output truncate is called with two parameters:

...
open(u'/blh/long_name', 32769)
getxattr(u'/blh/long_name', u'security.capability')
getxattr(u'/blh/long_name', u'security.capability')
removexattr(u'/blh/long_name', u'security.capability')
truncate(u'/blh/long_name', 0)
getattr(u'/blh/long_name', None)
Checking if global attr: blh
getxattr(u'/blh/long_name', u'security.capability')
getxattr(u'/blh/long_name', u'security.capability')
...

[blazk]

ok, glad to hear truncate() is now working for you! Does it mean your variable editing also works? (unfortunatelly I cannot get past the "not permitted in define mode" error" to test it).

[dvalters]

Does it mean your variable editing also works?

Unfortunately no :( There are no errors in debug mode or with saving/editing from vim, it just doesn't appear to change the file

[dvalters]

It is working for attributes now, I will see if I can figure out the issue with variables this evening

[dvalters]

Superseded by #42 after merging #41