search

dvalters / fuse-netcdf

ESoWC project to develop a Python utility to mount NetCDF files as a file-system in user space. (FUSE)
Other
8 stars 3 forks source link

Deleting variables #5

Open dvalters opened 6 years ago

dvalters commented 6 years ago

Spotted by Blazej in the meeting on 15/06:

netCDF API does not actually allow deleting variables from a netCDF file

http://unidata.github.io/netcdf4-python/#netCDF4.Variable

nco/pynco gets around this by creating a new file without the old variable. But this is not likely to work very easily with fusepy as you would have to unmount the old netcdf file and remount the new one every time you deleted a variable.

So we agreed to remove this as a feature in the acceptance criteria

blazk commented 6 years ago

@dvalters maybe an idea for for the future?:

In the .unlink() operation, instead of deleting a variable, rename it e.g. by adding some sort of special prefix. This prefix would hide it from FUSE so it would look like variable was deleted. (or maybe add the variable to some sort of list of pending deletions & hide it from FUSE? same thing really..)

When unmounting a file, check if there are any such "hidden" variables. If yes - create a copy of the dataset, excluding these hidden variables. Deferring dataset copying to the unmount time would be slightly more efficient than unmounting/copying/mounting the dataset every time a variable is deleted.