KeyError: 'sweep_data' when opening sigmet file

[aladinor]

• xradar version:0.1.0
• Python version: 3.9
• Operating System: Ubuntu 22.04

Hi Everyone,

I am trying to open a Sigmet file using Xradar; however, I got a KeyError. This is the code I am using

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import fsspec
import xradar as xd

def data_accessor(file) -> str:
    """
    Open AWS S3 file(s), which can be resolved locally by file caching
    """
    return fsspec.open_local(f'simplecache::s3://{file}', s3={'anon': True}, filecache={'cache_storage': '/tmp/radar/'})

def main():
    file = 's3-radaresideam/l2_data/2023/07/16/Tablazo/TAB230716015003.RAW7HD3'
    dt = xd.io.open_iris_datatree(data_accessor(file))
    pass

if __name__ == "__main__":
    main()

And this is the output:

  File "C:\Users\alfonso8\Miniconda3\envs\xradar\lib\site-packages\xarray\core\indexing.py", line 535, in __array__
    return np.asarray(array[self.key], dtype=None)
  File "C:\Users\alfonso8\Miniconda3\envs\xradar\lib\site-packages\xradar\io\backends\iris.py", line 3747, in __getitem__
    return indexing.explicit_indexing_adapter(
  File "C:\Users\alfonso8\Miniconda3\envs\xradar\lib\site-packages\xarray\core\indexing.py", line 826, in explicit_indexing_adapter
    result = raw_indexing_method(raw_key.tuple)
  File "C:\Users\alfonso8\Miniconda3\envs\xradar\lib\site-packages\xradar\io\backends\iris.py", line 3744, in _getitem
    return self.datastore.ds["sweep_data"][self.name][key]
KeyError: 'sweep_data'

Not sure if this is a corrupted file or a bug.

Please let me know what you think.

Cheers,

[kmuehlbauer]

@aladinor Sorry for the delay here. The issue is with a broken sweep inside this multisweep dataset. The 8th sweep contains broken data, which leads to this error you are seeing. The structure header looks like:

OrderedDict([('structure_header', OrderedDict([('structure_identifier', 1), ('format_version', 1), ('bytes_in_structure', 65537), ('flag', 1)])), ('sweep_start_time', datetime.datetime(1, 1, 1, 18, 12, 17, 1000)), ('sweep_number', 1), ('number_rays_per_sweep', 1), ('first_ray_index', 1), ('number_rays_file_expected', 1), ('number_rays_file_written', 1), ('fixed_angle', 0.0054931640625), ('bits_per_bin', 1), ('data_type', 1)])

which doesn't make any sense to me. I've no idea if that is just missing some bytes in between or whats going on there.

If you just omit the broken sweep the data is imported as usual.

# do not read 8th sweep
dt2 = xd.io.open_iris_datatree("TAB230716015003.RAW7HD3", sweep=[0,1,2,3,4,5,6,8])

[aladinor]

Thanks @kmuehlbauer for your help. I will close this issue since this is not a xradar bug. However, I'd like to know if there is a chance to add sort of more information about this error (like sweep n is corrupted). This is just an idea.

[kmuehlbauer]

@aladinor I have a fix which I can submit the next days.

With this fix the broken sweep is loaded somehow as single empty ray. But it might only work for this particular error.

[aladinor]

In that case, everything will be fine. Thanks @kmuehlbauer!