observingClouds / xbitinfo

Python wrapper of BitInformation.jl to easily compress xarray datasets based on their information content
https://xbitinfo.readthedocs.io
MIT License
54 stars 22 forks source link

[Bug] `get_keepbits` returning a negative integer value for `.grib` datasets #269

Open fnhirwa opened 8 months ago

fnhirwa commented 8 months ago

Description

Was trying the xbitinfo and following the tutorial presented and when compressing the .grib datasets it seems that the compression fails as the get_keepbits function returns negative values.

What I Did

import xbitinfo as xb
import xarray as xr

dataset = xr.tutorial.load_dataset("era5-2mt-2019-03-uk.grib").astype("float32")
info_p_bit = xb.get_bitinformation(dataset, dim="longitude", implementation="python")
keepbits = xb.get_keepbits(
    info_p_bit, 0.99
)  # returns negative number due to lossy compression
ds_bitrounded = xb.xr_bitround(
    dataset, keepbits
)  # raises a warning as keepbits is negative
sarthakvyas16 commented 1 month ago

i want to fix this issue please assign me this issue.

observingClouds commented 1 month ago

@sarthakvyas16 before assigning you to the issue it would be good to hear your ideas and discuss them.

sarthakvyas16 commented 1 month ago

ok let me explain. First of all i m review the implementation of get_keepbits. Understand its expected input, output, and the logic that leads to the computation of the return value after that Check the input being fed into get_keepbits. Ensure that the .grib datasets are formatted correctly and do not contain any corrupted or unexpected values and then ensure that all variables used in the calculations are of the correct data type. Sometimes, implicit type conversions can lead to unexpected negative values and tracing the calculation which help us to identify where the negative entity is originating. and many more things which use to sortout bug.If you are satisfy by my Point of view , then assign me this issue.