Not an issue just a question

[Boeblahoebla]

Hi there

Nice product you've got there. I was scouting through the code & I was wondering if there was a specific reason to leverage on the DWD tables for BUFR decoding.

Were the bufr tables provided by meteofrance not enough by any chance? I myself have also been curious about the meteofrance opendata, but the documentation seems very convoluted & hard to come by... It's also far from incomplete & doesn't provide all the information you would need.

I could also see in one of the files: Python_files/nlr_importdata.py that you used an offset for the meteofrance radardata. I was wondering how you came across this, as I can't seem to find anything about it in the documentation, nor can I find it in a BUFR files when decoding one.

I haven't been able to run the install of this projecton my mac though, so I can't run it & put breakpoints in my editor to see the flow of your application. Having problems with tensorflow & the install seems to break on it.

I would have liked to see how you actually downloaded the MF files because if you look at the api documentation of meteofrance, there is 0 mention of downloading archived data in their swagger. It all seems more convoluted & confusing than it should... Especially levaraging on BUFR, which is pretty archaic & less prone to standardisation (even if BUFR is actually intended as such).

Which is also the reason why DWD has moved to h5 instead of their BUFR files, although they apparently still provide the option to use it if I'm not mistaking.

When I download the Meteofrance BUFR files & decode them, I only get 1 product, reflectivity, while the PAM & PAG files normally contain more. I can't seem to find where (how) to get the other products as CC, ZDR, Advection (PAM) & radial velocity (PAG).

I was hoping your codebase could provide a clue. Keep up the good work! Seems like a major undertaking what you're doing. It's quite impressive

[Bram94]

Hi,

Thank you. The reason to use DWD's BUFR tables is simply a combination of "it works", and I used them already for DWD's BUFR files themselves.

And MF's documentation is indeed still limited. But I know they currently only provide real-time data, archived data is not yet online available. I asked them about this, and they answered me that this might change in the longer-term (2025/2026).

Regarding BUFR decoding: I also had to update my custom BUFR decoder for containing multiple products per file. You can find this custom decoder here: https://github.com/Bram94/numpy_bufr.git. In the case of PAM files the last product can't be decoded, but it's also not needed in my case (some advection IIRC). You can look at the function get_file_content in my MF BUFR decoder for how I deal with that.

Regarding NLradar install failure: Issues with tensorflow among others arise when using a too recent Python version. I recommend version 3.8, since that's what I use myself. Reasons for not using more recent version are issues with vispy and tensorflow, that I need to tackle at some point.

[Boeblahoebla]

Hi there

Thx for the reply. I took a look at the numpy bufr tool. It looks cool & seems to work for PAG files. I haven't tried for the PAM ones yet.

To circle back to my previous entry about the Python_files/nlr_importdata.py file. I can see that there are offsets declared for various (if not all the) products. The values are rather hardcoded so I assume they are not the result of a calculation done using the (meta)data of the MF bufr files.

Where did you find or how did you come across these? For example: the reflectivity offset of -10.5 is rather peculiar. All values in the data are rounded integers, which would make all the reflectivity values something like xx.5. I can't seem to find any mention about it in the MF documentation either.

I looked into the data when using the "numpy bufr" tool as well, & looked for an offset value, but the data doesn't provide it. Perhaps it's somewhere hidden in the cryptic documentation of MF which I might have looked over? Admittedly I have to say that I'm not an expert in radar-data generation, so forgive the (possibly stupid) questions.

The only thing I can think of is that these offsets are actually common across either types of radar dishes or is part of a standard across the OPERA network, but I don't seem to find these offsets in the german BUFR files either (you also don't seem to fetch it out), so my theory could be flawed. Perhaps the offset is available in an HDF5 file & instead of reading 2 files to fetch in the value in the HDF5, you hardcoded it to save time / processing? Just guessing here.

[Bram94]

Hi there,

I got these hardcoded values from their documentation, but they updated it recently, and it does now not include them anymore. I still could find one of the original descriptions though: https://donneespubliques.meteofrance.fr/client/document/descriptif_donnees_multipolarisees_pam_269.pdf. I suppose this information is also somehow contained in the BUFR files, but I don't remember the details about this. At least I probably had some good reason for opting for hardcoded values, I think it was notably easier to work with.