Open veenstrajelmer opened 9 months ago
Just checking @veenstrajelmer, does https://github.com/pydata/xarray/pull/8713 fix this issue?
I do not think so, this is about (lack of) precision of scale_factor attrs and is unrelated to {"_FillValue":np.nan} in the netcdf file attributes. I found the list of issues @Huite and myself collected a while ago. They are listed in this issue: https://github.com/Deltares/dfm_tools/issues/490. This one also lists another nan-fillvalue issue (https://github.com/Deltares/xugrid/issues/125), which we closed as "wont do". I guess it makes sense to do the same with this issue.
@JoerivanEngelen @veenstrajelmer Just stepping by from https://github.com/pydata/xarray/pull/8713. Correct, that PR won't fix this particular issue. But I'm aiming for solving the default fillvalue issue over at xarray, too. This is moving in small steps, though.
@kmuehlbauer thank you for clarifying this. We were considering to fix it in this package (prevent writing of _Fillvalue:np.nan
attributes and add encoding of default fillvalues per datatype if no _FillValue is supplied, both in line with the cf-convention. Do you mean that this behaviour will in the future be taken care of by xarray? In that case we can indeed consider to temporarily put it in xugrid to anticipate future behaviour.
I noticed that xugrid writes nan fillvalue attributes for variables without fillvalue defined. After a quick assessment I see that this is xarray behaviour and it is also happening in their code. A nan fillvalue does not make sense to me and it causes issues in some cases, like this one. In dfm_tools there is a workaround in place that pops nan fillvalue attrs upon reading, but I believe we should avoid writing them. It might be related to xarray not decoding default fillvalues if no fillvalue attribute is present. Either way, let's discuss what is expected/desired behaviour here.
The code below reads a D-FlowFM output which has no nan fillvalue attributes. When writing it with
.to_netcdf()
there are nan fillvalue attributes. I have tried this also with xugrid testdata, but those datasets already contain nan fillvalue attributes upon reading: