jdkloe / pybufr-ecmwf

a python module that allows reading and writing BUFR formatted files, where BUFR stands for Binary Universal Form for the Representation of meteorological data.
Other
29 stars 12 forks source link

Fortran error #17

Closed DivinytyToffee closed 5 years ago

DivinytyToffee commented 5 years ago

Hi, I tryed read the bufr and writing data in arrays. Early this method work, but now that retur error: Sorry, call to bufrex failed, reported fortran error(s)". And when I call get_next_msg() method I get EOFError. Why I do wrong, or how this fix?

    bufr = BUFRReader(path_to_file)

    for msg in bufr:
        for msg_or_subset_data in msg:
            all_data = np.dstack((msg_or_subset_data.names,
                                  msg_or_subset_data.data,
                                  msg_or_subset_data.units))[0]
jdkloe commented 5 years ago

Hi, thanks for your report. From your example code I do not see a clear problem. Could you provide a small test data file that triggers the problem so I can test it on my side?

DivinytyToffee commented 5 years ago

@jdkloe, I found a source of mistake, problem was in some files. But I have a one more question, how I can decode BUFR message from bytearry? Like this:

bdata = BytesIO()
        try:
            ftp.retrbinary('RETR ' + file, bdata.write)
            bdata.seek(0)
            data = bdata.read()
            BUFRReader(data)
jdkloe commented 5 years ago

There is no dedicated code in the interface to handle this, so thanks for your suggestion, I will add it when I can find some time to work on this again. And yes, it is possible to directly insert bytes for decoding, but it involves filling and using methods of an instance of the lower level RawBUFRFile class, and creation of a dummy file. This works for me:

#!/usr/bin/python3
import os
import numpy as np
from io import BytesIO
from pybufr_ecmwf.bufr import BUFRReader
from pybufr_ecmwf.raw_bufr_file import RawBUFRFile

fn = 'test/testdata/Testfile.BUFR'
bdata_in = open(fn, 'rb').read()
buffer = BytesIO(bdata_in)

dummyfile = 'dummyfile'
if not os.path.exists(dummyfile):
    with open(dummyfile, 'wb') as df:
        df.write(b'\n')

bdata = buffer.getvalue()
bufr = BUFRReader('dummyfile')
bufr._rbf = RawBUFRFile()
bufr._rbf.filesize = len(bdata)
bufr._rbf.bufr_fd = 'dummy' # it most be different from None
bufr._rbf.data = bdata
bufr._rbf.split()
bufr.num_msgs = bufr._rbf.get_num_bufr_msgs()

for msg in bufr:
    for msg_or_subset_data in msg:
        n = msg_or_subset_data.names,
        d = msg_or_subset_data.data,
        u = msg_or_subset_data.units
        print(n, d, u)
DivinytyToffee commented 5 years ago

Thanks! It's work. But would I correctly understanding, what me not necessary call close() as bufr.close(), just where I call it fall with exception: "'str' object has no attribute 'close'"?

jdkloe commented 5 years ago

The file reading is bypassed by this workaround, so no need for that I think. Only this dummy file is opened so I dont expect large memory usage. If you plan to do this decoding in a loop and are worried about opening this dummy file too many times, you could add the following close command just to be sure:

bufr.close()