For Delta Dash where nuding files include both obs and hycom, file size is a big issue leading to hundreds of Gb.
At the same time a casual test based on zipping (often very close) suggests that compression would reduce file size by 95%. It is likely that using compression with netcdf will produce the same, it usually does. How does this slow down the creation and writing out of the file? Does compression slow down schism?
Example: script produces 25Gb for a a 60d nudging job and it becomes about 1Gb. The entire year would be 6Gb, which would be entirely satisfactory as far as storage and movement of files.
For Delta Dash where nuding files include both obs and hycom, file size is a big issue leading to hundreds of Gb.
At the same time a casual test based on zipping (often very close) suggests that compression would reduce file size by 95%. It is likely that using compression with netcdf will produce the same, it usually does. How does this slow down the creation and writing out of the file? Does compression slow down schism?
Example: script produces 25Gb for a a 60d nudging job and it becomes about 1Gb. The entire year would be 6Gb, which would be entirely satisfactory as far as storage and movement of files.