Change `check_system_memory` call for parallel_stereo.

[jomey]

Change querying the available memory for parallel stereo by using the information from '/proc/meminfo' instead of the free command line call. The latter threw an error on Pleiades and using the proc information is also in line with Common.cc#397.

This will still only work on Linux machines.

[jomey]

To give a bit more context, I was getting the error message:

Warning: Error checking system memory, skipping the memory test!

in my recent parallel_stereo runs. Once I changed the python script to the above, I was no longer getting the warning.

[oleg-alexandrov]

That this does not work and was not tested for the Mac is a challenge.

[jomey]

I can add a similar piece of logic as in Common.cc

Would you prefer that?

[oleg-alexandrov]

Can it be tested on both Linux and the Mac? Then at some point when I am back from vacation I could pull this in.

I am a bit embarrassed of the code you point to. Doing system calls in C++ code is just downright ugly. The alternative was though to either do some kind of wrapper around every tool or to find some C calls for all those things we wanted to log.

[jomey]

I will have a look at a portable solution for both platforms.

I agree that it feels 'hackish' to do system calls from C/C++. At the same time it reduces the need to write your own and/or pull in another library for a call that simple.

Down the road it might be nice to have a central way of doing system checks from within ASP that is accessible via C/C++ and Python. Just a dream though ....

[jomey]

To support a Mac it is a little more involved.

I found this approach (after a lot of help from the inter web). Getting the current memory info:

~:$ vm_stat 
Mach Virtual Memory Statistics: (page size of 4096 bytes)
Pages free:                             1165528.
Pages active:                           1559673.
Pages inactive:                          294917.
Pages speculative:                       341968.
Pages throttled:                              0.
Pages wired down:                        767935.
Pages purgeable:                         379446.
"Translation faults":                 663649407.
Pages copy-on-write:                   30333333.
Pages zero filled:                    325994816.
Pages reactivated:                      1541713.
Pages purged:                           8458583.
File-backed pages:                       788881.
Anonymous pages:                        1407677.
Pages stored in compressor:              626321.
Pages occupied by compressor:             64085.
Decompressions:                         1627707.
Compressions:                           3744781.
Pageins:                               17352652.
Pageouts:                                  5389.
Swapins:                                 353943.
Swapouts:                                434867.

My local machine had ~7 GB available at the time of querying this and to get there you would have to add up free, inactive, and speculative:

~:$ bc
bc 1.06
Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'. 
(1165528 + 294917 + 341968)*4096/1024/1024
7040

[ShashankBice]

Hi, Will psutil python package be of help here ? On Linux, this is what I get:

import psutil
mem = psutil.virtual_memory()
cpu = psutil.cpu_count()

>>> print(mem) svmem(total=66968354816, available=39578419200, percent=40.9, used=25668194304, free=14981206016, active=15446773760, inactive=17018105856, buffers=506470400, cached=25812484096, shared=514080768, slab=15116365824) >>> print(cpu) 16 From documentation, it looks like this should work on mac as well, although I do not have one to test.

[jomey]

The package can be an option, not sure if that is a good solution to add this dependency for one call.

Since there are some conflicts on this branch and I have no time to look into this, I am closing this. Hopefully some other solution can be found.