astrogo / fitsio

fitsio is a pure-Go package to read and write `FITS` files
BSD 3-Clause "New" or "Revised" License
53 stars 24 forks source link

Reusing the I/O buffer when decoding a header unit would be beneficial #31

Closed airnandez closed 6 years ago

airnandez commented 6 years ago

The current implementation of readBlock allocates a new memory buffer every time a new block of 2880 bytes is to be read (see here). This generates additional unnecessary work to the garbage collector when decoding thousands of files in a single execution of a program using astrogo/fitsio.

This situation could be improved by associating a buffer of the appropriate size to a streamDecoder and reuse that buffer for reading each block of the same FITS file.

Alternatively, a single buffer could be allocated by the DecodeHDU() method and that buffer could be reused to perform all the necessary read operations while decoding the entire HDU.

sbinet commented 6 years ago

thanks for the report. I went with the alternative :)

airnandez commented 6 years ago

Excellent. Thanks !