hfgolino / EGAnet

43 stars 9 forks source link

Incorrect available memory estimates on Linux #140

Closed atomashevic closed 1 year ago

atomashevic commented 1 year ago

Hi,

I've tried to run a dynEGA model upon installing EGAnet version: 2.0.2 and it fails because it provides incorrect estimation of free memory needed to run the model.

I tried to run this model:

dega.add.all.0.indpop <- dynEGA(
    data = data.add[,-8],
    n.embed = 10,
    tau = 1,
    delta = 1,
    level = c('individual','population'),
    id = 7,
    use.derivatives = 0
)

And here's the memory report from RStudio along with the error code

Screenshot from 2023-10-17 10-15-36

I think the error comes from the assumption that free on Linux returns values in bytes, instead the default is in kilobytes (actually it's kibibytes, but should be the same thing): https://man7.org/linux/man-pages/man1/free.1.html

Also, I think you should use the number stated in the available column. This corresponds to the "Free" section in RStudio's memory report. In the case above the available columns says 7444920.

I tested the following change and it works for me. Changed lines 501-502 of helpers.R:

    # Get free values (Linux reports in kB)
    bytes <- as.numeric(info_split[2, info_split[1,] == "available"]) * 1024

My session info:

R version 4.2.2 Patched (2022-11-10 r83330)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 23.04

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.21.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] TSEntropies_0.9 zoo_1.8-11      EGAnet_2.0.2   

loaded via a namespace (and not attached):
[1] compiler_4.2.2  tools_4.2.2     rstudioapi_0.14 grid_4.2.2      lattice_0.20-45
AlexChristensen commented 1 year ago

Thanks for pointing this issue out, @atomashevic!

I must have miss counted the digits when first checking on Linux. I switched to "available"

I used 1000 rather than 1024 to be on the conservative side. Otherwise, there can be an error that is unintelligible and the purpose of this error is to be a bit more specific of why there is an error.

Closing as resolved