jdkloe / pybufr-ecmwf

a python module that allows reading and writing BUFR formatted files, where BUFR stands for Binary Universal Form for the Representation of meteorological data.
Other
29 stars 12 forks source link

BUFRReaderBUFRDC.get_value() method - incorrect values #11

Closed jintaglee closed 7 years ago

jintaglee commented 7 years ago

Hello,

I have a Bufr file from which I want to pick out the values of a specific descriptor. However, it appears the values look wrong. In order for this problem to be reproduced I have uploaded a sample data file in the following anonymous FTP server,

Name of FTP server: ftp.bom.gov.au Login name: anonymous Password: (not required) Location of Bufr file: /anon/home/cawcr/10day/jinlee/HIMCSR_1.bufr

Following is the set of commands I used to pick the values of the descriptor number, 96 ('BRIGHTNESS TEMPERATURE') from the message=2 and subsets=1, 2, etc.,

bfr=BUFRReader('HIMCSR_1.bufr') bfr.get_next_msg() bfr.get_next_msg() bfr.get_value(96,1)

I used ECMWF's Metview as an independent check on the values. I checked subsets of other messages and there there doesn't appear to be any discrepancy between the values printed out by get_value() and Metview. So the problem seems to be for message=2 and some of its subsets.

If you can have a look I would appreciate it.

Regards,

Jin

jdkloe commented 7 years ago

Hi Jin, thank you for reporting your problem. I succesfully downloaded your sample BUFR file and if I decode it I can reproduce the same output that you get. However, I am not convinced the output is wrong. I also decoded this BUFR message using the latest ecCodes version (v2.0.0). I executed: ./bufr_dump -j f -t /path/to/HIMCSR_1.bufr > tmp and on line 7347 of the generated tmp file I find: "null, 291.3, 292.2, null, null, ... etc" for "brightnessTemperature". Also if I look at a longer range of subsets the values produced by ecCodes and pybufr_ecmwf exactly match, so I am convinced I am looking at the same bufr message with both tools.

How sure are you about the Metview output? Maybe it uses a different indexing? Please note that pybufr_ecmwf starts counting at 0 for elements in a subset, but starts counting at 1 for subsets. Its confusing, I know, and I hope to get this cleaned-up in a next version.

Anyway, it may be easier to get the data using the expression: data = bfr.msg.get_values_as_2d_array()

jintaglee commented 7 years ago

Hi Jos,

Sorry for this late reply. I was tied up with an operational matter and didn't have the chance to have a look at the problem.

You're right: pybufr-ecmwf is decoding the Bufr message correctly. I checked using EMOS library and the values are identical to those from pybufr-ecmwf. So this means Metview is wrong (?!), which I find difficult to believe since it is a software package used by quite a lot of people at ECMWF. I will have to contact ECMWF and find out if it's Metview or it's something I'm doing wrong.

Thank you, Jos. I will keep you informed of what I find out from ECMWF.

jdkloe commented 7 years ago

Hi Jin, thanks for your message. Please keep me posted when you find out what was the issue here. I also don't understand why Metview would give different results, since it is build on the same bufrdc library from ECMWF.

jdkloe commented 7 years ago

For now, closing this issue.