spider-gazelle / bindata

BinData - Parsing Binary Data in Crystal Lang
MIT License
48 stars 5 forks source link

How to skip N bytes when reading IO? #4

Open DRVTiny opened 4 years ago

DRVTiny commented 4 years ago

I'm trying to read GRIB2 files (storm and weather predictions) using official NCEP NOAA documentation. Say, i need to read Section 1: https://www.nco.ncep.noaa.gov/pmb/docs/grib2/grib2_doc/grib2_sect1.shtml , but this section contains no useful data for me and all i need - is to read first 4 bytes of this section to something like "section1_size" property and skip this "section1_size bytes" totally to read something more valuable after the section1. How can i achieve this goal using bindata shard? Thank you (and sorry for my awful english :)) !

mattrberry commented 4 years ago

I don't think there's actually a way to skip data and not read it into some field on your class yet. Instead, you could do something like

class GRIB2 < BinData
  endian big
  bytes :skip, length: ->{ 4 }
end

@stakach correct me if I'm wrong, but that's how I would probably do it. Could be useful to add a skip macro. I can do this in the next couple days if you're interested.

stakach commented 4 years ago

Yeah bytes :skip, length: ->{ 4 } is how i would do it Not sure it requires a skip macro as typically you'd want this data (even if you are ignoring it) to re-serialize the structure.

mattrberry commented 4 years ago

If you're just consuming from an API then there's no need to re-serialize the structure. I imagine there are other use-cases as well that wouldn't necessarily require re-serialization, so a skip macro might be useful in those cases.

stakach commented 4 years ago

@mattrberry will accept a pull request if you think it's worth implementing.

mattrberry commented 4 years ago

:+1:

DRVTiny commented 4 years ago

If you're just consuming from an API then there's no need to re-serialize the structure. I imagine there are other use-cases as well that wouldn't necessarily require re-serialization, so a skip macro might be useful in those cases.

Exactly. For example in my case - i don't need to reassemble GRIB2 file, i need to make weather prediction "in-point" using grid data of GFS prediction model :) And there are many cases where proper reading and verifying data in binary format is only needed.