Cannot allocate memory

[davidyoung8906]

Hi! I got an "cannot allocate memory" error while running Mudflow with default data. I am running it on a machine with 16 GB memory. Can you let me know the expected environment to run the tool? You can find the error message as follows:

weiyang3@ase-Precision-T1700:~/SharedSpace/WeiYang/mudflow-statistical-analysis/scripts$ Rscript make_all.R WARNING - no config file specified, using default INFO - config loaded conf Loading required package: methods [1] "/media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/" Loading required package: foreach Loading required package: iterators Loading required package: parallel file name: /media/ase/WeiYang/mudflow-statistical-analysis/data/mudflow_paper.csv data 17897 3735 m: 15097 b: 2800 [1] TRUE *dprep* Error in { : task 1 failed - "cannot popen 'cd /media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/1/orca/ && /media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/1/orca/dprep-linux /media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/1/orca/pre/ACCOUNT_INFORMATION_test.data /media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/1/orca/pre/ACCOUNT_INFORMATION_test.fields /media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/1/orca/bin/ACCOUNT_INFORMATION_test.bin -snone', probable reason 'Cannot allocate memory'" Calls: %dopar% -> <Anonymous> Execution halted Warning message: system call failed: Cannot allocate memory

Thanks!

[vitaliiavdiienko]

Hi,

the reason is in the fact that we run an analysis in parallel in a big amount of data. That is why R has run out of memory.

The solution is to reduce the number of parallel instances by playing with the n_cores paramater in the conf.R file. Try to make itn_cores=2. This should help.

Best Regards, Vitalii

[davidyoung8906]

Thanks! Vitalii. After solving the memory issue. I met another issue while running MUDFLOW. Here is the output:

weiyang3@ase-Precision-T1700:~/SharedSpace/WeiYang/mudflow-statistical-analysis/scripts$ Rscript make_all.R
WARNING - no config file specified, using default
INFO - config loaded conf
Loading required package: methods
[1] "/media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/"
Loading required package: foreach
Loading required package: iterators
Loading required package: parallel
file name: /media/ase/WeiYang/mudflow-statistical-analysis/data/mudflow_paper.csv
data 17897 3735
m:  15097 b:  2800
[1] FALSE
Warning messages:
1: In dir.create(dest, recursive = TRUE) :
  '/media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/1/orca' already exists
2: In dir.create(dest, recursive = TRUE) :
  '/media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/2/orca' already exists
3: In dir.create(dest, recursive = TRUE) :
  '/media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/3/orca' already exists
4: In dir.create(dest, recursive = TRUE) :
  '/media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/4/orca' already exists
5: In dir.create(dest, recursive = TRUE) :
  '/media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/5/orca' already exists
6: In dir.create(dest, recursive = TRUE) :
  '/media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/6/orca' already exists
7: In dir.create(dest, recursive = TRUE) :
  '/media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/7/orca' already exists
8: In dir.create(dest, recursive = TRUE) :
  '/media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/8/orca' already exists
9: In dir.create(dest, recursive = TRUE) :
  '/media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/9/orca' already exists
10: In dir.create(dest, recursive = TRUE) :
  '/media/ase/WeiYang/mudflow-statistical-analysis/mudflow_paper_10parts/10/orca' already exists
dprep-linux: bfile.cpp:508: void BFile::seek_position (int): Assertion `pos < records_' failed.
Aborted (core dumped)
dprep-linux: bfile.cpp:508: void BFile::seek_position (int): Assertion `pos < records_' failed.
Aborted (core dumped)
dprep-linux: bfile.cpp:508: void BFile::seek_position (int): Assertion `pos < records_' failed.
Aborted (core dumped)
dprep-linux: bfile.cpp:508: void BFile::seek_position (int): Assertion `pos < records_' failed.
Aborted (core dumped)
dprep-linux: bfile.cpp:508: void BFile::seek_position (int): Assertion `pos < records_' failed.
Aborted (core dumped)
dprep-linux: bfile.cpp:508: void BFile::seek_position (int): Assertion `pos < records_' failed.
Aborted (core dumped)
dprep-linux: bfile.cpp:508: void BFile::seek_position (int): Assertion `pos < records_' failed.
Aborted (core dumped)
dprep-linux: bfile.cpp:508: void BFile::seek_position (int): Assertion `pos < records_' failed.
Aborted (core dumped)
dprep-linux: bfile.cpp:508: void BFile::seek_position (int): Assertion `pos < records_' failed.
Aborted (core dumped)
dprep-linux: bfile.cpp:508: void BFile::seek_position (int): Assertion `pos < records_' failed.
Aborted (core dumped)
dprep-linux: bfile.cpp:508: void BFile::seek_position (int): Assertion `pos < records_' failed.
Aborted (core dumped)
*weights*
Error in { : task 9 failed - "no lines available in input"
Calls: %dopar% -> <Anonymous>
Execution halted

Can you let me know what could cause the problem?

Thanks a lot!

[davidyoung8906]

Hi! Vitalii. Can you help on the issue?

[vitaliiavdiienko]

Hi,

did you try it on other machine? we didn't experience the same error.

Can you maybe provide us with steps to reproduce or data if the proposal above will not work?

[davidyoung8906]

Thanks! I resolved the issue. It should be machine-specific problem

[rarabzad]

Hi,

Unfortunately I am also getting a similar issue on my machine but not with Modflow. I am trying to parallelize for a hydrology model but I get the following error:

"Error in { : task 7 failed - "no lines available in input""

everything works fine as long as I am not parallelizing. Any idea?? Thanks