Memory management with multiple messages

[odrans]

Hi @nawendt ,

thanks a lot for this nice package, very useful!

I used it on ERA5 data and noticed when running grib_get_message with multiple messages, for example

gm_multi <- grib_get_message(g, c(1, 2, 3))

that the memory usage associated with the R process increases significantly, by orders of magnitude beyond the actual size of the gm_multi object. This does not happen when running the function iteratively on single messages, e.g.

gm_multi <- plyr::llply(c(1, 2, 3), function(idx) gribr::grib_get_message(g, idx))

Removing all objects and running garbage collector in R doesn't really decrease the memory use associated with the process.

I am not very familiar with C nor with the associated eccodes functions, but noticed in gribr_grib_get_message that codes_handle_delete(h) is only called when messagesLength = 1. Could that be one reason for the memory not being deallocated in C?

Thanks, Odran

[nawendt]

Thanks. I will try and look at this soon. I would think I can reproduce this with any GRIB file. If not, I may need an example file from you.

[nawendt]

Confirmed the leak with valgrind. Should be fixed now.