Closed finneyb closed 8 years ago
Thanks for submitting this issue. If that is the only test failing, then it is very likely that your build is correct. I'll therefore assume this is not a critical issue for you and it will be several days before I have a chance to investigate this further.
I am able to recreate the same behavior on a Travis-CI Ubuntu linux environment. Also, it seems that this test isn't even attempted under OS X. Just curious what the reason is for this.
It's not attempted on OS X because it failed on OS X due to an incompatibility with the FFT routines. As mentioned in the other thread, I'd be glad to switch to other FFTs as the default if it's something you'd like to try.
@finneyb: Could you please re-run ctest
and add the --verbose
flag? (i.e. ctest --verbose
) and also paste the output into a fenced code block so that it doesn't get interpreted as markdown?
< code >
@afanfa would you mind taking a quick look at this error message on the Travis-CI build? I'm guessing this is due to an issue with the FFT library, but I'm not sure... I was reading that some people had problems with /dev/urandom on Travis-CI as well, but according to my testing that is not the issue. Here is the log line where the error message starts: https://travis-ci.org/zbeekman/opencoarrays/jobs/96362389#L1301
@afanfa & @rouson my testing on Travis-CI also seems to indicate that libfft_sse.a
is always used for linking no matter what -DLEGACY_ARCHITECTURE=
is set to... I'm can try to investigate more, but for now this is challenging, since I can only build locally on a Mac.
sorry, forgot the code block. Here it is again.
Start testing: Dec 11 16:03 PST
----------------------------------------------------------
1/22 Testing: initialize_mpi
1/22 Test: initialize_mpi
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/init_register/initialize_mpi"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"initialize_mpi" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 0.05 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"initialize_mpi" end time: Dec 11 16:03 PST
"initialize_mpi" time elapsed: 00:00:00
----------------------------------------------------------
2/22 Testing: register
2/22 Test: register
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/init_register/register"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"register" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 0.04 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"register" end time: Dec 11 16:03 PST
"register" time elapsed: 00:00:00
----------------------------------------------------------
3/22 Testing: register_rename_me
3/22 Test: register_rename_me
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/init_register/register_rename_me"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"register_rename_me" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 0.04 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"register_rename_me" end time: Dec 11 16:03 PST
"register_rename_me" time elapsed: 00:00:00
----------------------------------------------------------
4/22 Testing: register_rename_me_too
4/22 Test: register_rename_me_too
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/init_register/register_rename_me_too"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"register_rename_me_too" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 0.04 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"register_rename_me_too" end time: Dec 11 16:03 PST
"register_rename_me_too" time elapsed: 00:00:00
----------------------------------------------------------
5/22 Testing: allocate_as_barrier
5/22 Test: allocate_as_barrier
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/init_register/allocate_as_barrier"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"allocate_as_barrier" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 1.04 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"allocate_as_barrier" end time: Dec 11 16:03 PST
"allocate_as_barrier" time elapsed: 00:00:01
----------------------------------------------------------
6/22 Testing: allocate_as_barrier_proc
6/22 Test: allocate_as_barrier_proc
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/init_register/allocate_as_barrier_proc"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"allocate_as_barrier_proc" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 1.04 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"allocate_as_barrier_proc" end time: Dec 11 16:03 PST
"allocate_as_barrier_proc" time elapsed: 00:00:01
----------------------------------------------------------
7/22 Testing: get_array
7/22 Test: get_array
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/send-get/get_array"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"get_array" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed
Test passed
<end of output>
Test time = 0.64 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"get_array" end time: Dec 11 16:03 PST
"get_array" time elapsed: 00:00:00
----------------------------------------------------------
8/22 Testing: send_array
8/22 Test: send_array
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/send-get/send_array"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"send_array" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed
Test passed
<end of output>
Test time = 0.88 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"send_array" end time: Dec 11 16:03 PST
"send_array" time elapsed: 00:00:00
----------------------------------------------------------
9/22 Testing: get_with_offset_1d
9/22 Test: get_with_offset_1d
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/send-get/get_with_offset_1d"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"get_with_offset_1d" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 0.04 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"get_with_offset_1d" end time: Dec 11 16:03 PST
"get_with_offset_1d" time elapsed: 00:00:00
----------------------------------------------------------
10/22 Testing: whole_get_array
10/22 Test: whole_get_array
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/send-get/whole_get_array"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"whole_get_array" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 0.05 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"whole_get_array" end time: Dec 11 16:03 PST
"whole_get_array" time elapsed: 00:00:00
----------------------------------------------------------
11/22 Testing: strided_get
11/22 Test: strided_get
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/send-get/strided_get"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"strided_get" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 0.05 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"strided_get" end time: Dec 11 16:03 PST
"strided_get" time elapsed: 00:00:00
----------------------------------------------------------
12/22 Testing: co_sum
12/22 Test: co_sum
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "4" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/collectives/co_sum_test"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"co_sum" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 0.13 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"co_sum" end time: Dec 11 16:03 PST
"co_sum" time elapsed: 00:00:00
----------------------------------------------------------
13/22 Testing: co_broadcast
13/22 Test: co_broadcast
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "4" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/collectives/co_broadcast_test"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"co_broadcast" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 0.10 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"co_broadcast" end time: Dec 11 16:03 PST
"co_broadcast" time elapsed: 00:00:00
----------------------------------------------------------
14/22 Testing: co_min
14/22 Test: co_min
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "4" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/collectives/co_min_test"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"co_min" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 0.10 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"co_min" end time: Dec 11 16:03 PST
"co_min" time elapsed: 00:00:00
----------------------------------------------------------
15/22 Testing: co_max
15/22 Test: co_max
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "4" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/collectives/co_max_test"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"co_max" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Correct real(c_double) co_max
Test passed.
<end of output>
Test time = 0.12 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"co_max" end time: Dec 11 16:03 PST
"co_max" time elapsed: 00:00:00
----------------------------------------------------------
16/22 Testing: syncall
16/22 Test: syncall
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "32" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/sync/syncall"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"syncall" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 7.31 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"syncall" end time: Dec 11 16:03 PST
"syncall" time elapsed: 00:00:07
----------------------------------------------------------
17/22 Testing: syncimages
17/22 Test: syncimages
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "32" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/sync/syncimages"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"syncimages" start time: Dec 11 16:03 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 8.33 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"syncimages" end time: Dec 11 16:04 PST
"syncimages" time elapsed: 00:00:08
----------------------------------------------------------
18/22 Testing: co_reduce
18/22 Test: co_reduce
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "4" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/unit/collectives/co_reduce_test"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"co_reduce" start time: Dec 11 16:04 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 0.15 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"co_reduce" end time: Dec 11 16:04 PST
"co_reduce" time elapsed: 00:00:00
----------------------------------------------------------
19/22 Testing: hello_multiverse
19/22 Test: hello_multiverse
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/integration/coarrayHelloWorld/hello_multiverse"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"hello_multiverse" start time: Dec 11 16:04 PST
Output:
----------------------------------------------------------
Greetings from image 1 of 2
Greetings from image 2 of 2
Test passed.
<end of output>
Test time = 0.04 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"hello_multiverse" end time: Dec 11 16:04 PST
"hello_multiverse" time elapsed: 00:00:00
----------------------------------------------------------
20/22 Testing: coarray_burgers_pde
20/22 Test: coarray_burgers_pde
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/integration/pde_solvers/coarrayBurgers/coarray_burgers_pde"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"coarray_burgers_pde" start time: Dec 11 16:04 PST
Output:
----------------------------------------------------------
Time = 0.60065795534754485
On image 2 u = 1.9805929082814393E-015 -2.5585980904689749 -3.1602423341809680 -2.7616422370249771 -2.2228610006371614 -1.6695096165091943 -1.1137114029330646 -0.55703975723770482
On image 1 u = -3.6062644769485318E-016 0.55703975723770405 1.1137114029330639 1.6695096165091936 2.2228610006371610 2.7616422370249762 3.1602423341809680 2.5585980904689771
Test passed.
<end of output>
Test time = 0.11 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"coarray_burgers_pde" end time: Dec 11 16:04 PST
"coarray_burgers_pde" time elapsed: 00:00:00
----------------------------------------------------------
21/22 Testing: co_heat
21/22 Test: co_heat
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/integration/pde_solvers/coarrayHeatSimplified/co_heat"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"co_heat" start time: Dec 11 16:04 PST
Output:
----------------------------------------------------------
Test passed.
<end of output>
Test time = 32.56 sec
----------------------------------------------------------
Test Pass Reason:
Required regular expression found.Regex=[Test passed.
]
"co_heat" end time: Dec 11 16:04 PST
"co_heat" time elapsed: 00:00:32
----------------------------------------------------------
22/22 Testing: coarray_navier_stokes
22/22 Test: coarray_navier_stokes
Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/integration/pde_solvers/navier-stokes/coarray_navier_stokes"
Directory: /usr/local/bin/opencoarrays/opencoarrays-build
"coarray_navier_stokes" start time: Dec 11 16:04 PST
Output:
----------------------------------------------------------
[proxy:0:0@madison] HYDU_create_process (/usr/local/bin/mpich-3.1.4/src/pm/hydra/utils/launch/launch.c:75): execvp error on file /usr/local/bin/opencoarrays/opencoarrays-build/src/tests/integration/pde_solvers/navier-stokes/coarray_navier_stokes (No such file or directory)
[proxy:0:0@madison] HYDU_create_process (/usr/local/bin/mpich-3.1.4/src/pm/hydra/utils/launch/launch.c:75): execvp error on file /usr/local/bin/opencoarrays/opencoarrays-build/src/tests/integration/pde_solvers/navier-stokes/coarray_navier_stokes (No such file or directory)
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 15397 RUNNING AT madison
= EXIT CODE: 255
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
<end of output>
Test time = 0.03 sec
----------------------------------------------------------
Test Fail Reason:
Required regular expression not found.Regex=[Test passed.
]
"coarray_navier_stokes" end time: Dec 11 16:04 PST
"coarray_navier_stokes" time elapsed: 00:00:00
----------------------------------------------------------
End testing: Dec 11 16:04 PST
@finneyb Ah, OK. I should have noticed earlier, you're experiencing a completely different issue than I am...
@finneyb can you please tell me the output of running uname -mps
on the machine you're running this on, as well as the command(s) you used to install open coarrays? (e.g. ./install.sh /usr/local/bin
or mkdir build && cd build && cmake .. && make && make install
etc.)
It looks like either there may be a problem in the logic of how the navier_stokes test is conditionally compiled and added to the list of tests in the CMakeLists.txt. (It links against an FFT library written directly in assembly, and only works on x86_64 Linux, so the CMake build tries to ensure that the OS and architecture will support the library before compiling and linking the test that requires it.)
Some secondary hunches as to where the problem could be are:
I'll try to look into this some more.
On 12/11/2015 4:37 PM, Izaak Beekman wrote:
@finneyb https://github.com/finneyb Ah, OK. I should have noticed earlier, you're experiencing a completely different issue than I am...
@finneyb https://github.com/finneyb can you please tell me the output of running |uname -mps| on the machine you're running this on, as well as the command(s) you used to install open coarrays? (e.g. |./install.sh /usr/local/bin| or |mkdir build && cd build && cmake .. && make && make install| etc.)
uname -mps Linux armv7l armv7l
installed open coarrays with ./install.sh /usr/local/bin/opencoarrays
Looks like the installer should just not install that test at all as this machine is not x86_64.
yes I'm a bit confused to why it is being attempted... digging around further.... For now it is certainly safe to ignore that error... the FFT library is not part of OCA just part of the tests... I'm working on fixing this now.
I can confirm that this test failure is due to the fact that the top level CMakeLists.txt has unintentionally diverged between the master branch and the 1.x branch (branch-1.0.0
) that the releases are based off of. This should be fixed in the next release. (I am working to resolve some unintentional differences soon, as I discussed on the phone with Damian.)
This line is the culprit: https://github.com/sourceryinstitute/opencoarrays/blob/branch-1.0.0/CMakeLists.txt#L137
and it should look like: https://github.com/sourceryinstitute/opencoarrays/blob/master/CMakeLists.txt#L174
@zbeekman Hi, I am sorry...I couldn't understand what exactly is this issue with the navier_stokes test. I suppose we fixed the issue involving 32-bit vs 64-bit version build. This test was supposed to run on only the 64-bit version Linux build and should be skipped for others. Does the previous patch not working on OS X?
@naveen-rn sorry if I confused you. This issue is now resolved. The problem was due to patches being applied to the master branch but not the 1.x release branch. On the 1.x the x86_64 check was applied when building the test but not when adding it. So on @finneyb's arm machine CMake was trying to run the test even though it wasn't built.
Some other unrelated problem is occurring on the TravisCI build (this is an Ubuntu virtual machine that reports itself as x86_64) in which a SIGBUS is being raised by attempting to access an invalid memory location during the coarray_navier_stokes test. My hunch is that this is due to a virtualization/architecture issue with the FFT library, but TBH I really don't know the cause for sure. (No compile/link time errors are raised and I can't seem to generate a call stack backtrack with line numbers) For now I disable the test when running on the Travis-CI VMs, but if someone could take a look and figure out the source of the problem I would certainly welcome that. Here is a link to a recent build exhibiting the problem: https://travis-ci.org/zbeekman/opencoarrays/jobs/96362389#L1301
Fixed by PR #49
In case it helps, you might download the Linux virtual machine that is available in the Sourcery Institute store: http://www.sourceryinstitute.org/store. There are instructions for importing and booting the virtual machine in the free virtualization package VirtualBox available at http://www.virtualbox.org. I will be uploading a slightly improved version of the virtual machine sometime soon -- hopefully tonight.
Great! I've actually been meaning, for quite a long time to dual boot my MPB, but haven't gotten around to it (for years now...) so I should really take you up on the offer. My one hesitation is that space is at a premium for me right now.
However, it seems that you don't have any issues running this test on your VM Linux instance, so I think the issues on Travis-CI with this test must be related to their specific virtualization environment.
You can store the vm on an USB flash drive. The vm itself expands to about 12 GB when installed. It has limited space so I generally mount a second virtual drive that occupies a bit under 12 GB. To play it safe, a flash drive with a capacity of 32 GB is probably best. Also, it's best if it uses USB 3 for speed.
The virtual drive is also online at the URL below and I can provide a link to it via email with instructions for mounting it if you like. Alternatively, I can ask my assistant to mail everything to you on a 32 GB flash drive if you like. If so, let me know.
I don't think my current MBP supports USB 3, but I'm not 100%. I also have a second drive installed (in place of the cd-dvd drive) so I could probably figure out a way to mount/install it there. I'd just like to catch up on a bunch of the stuff we spoke about on the phone before I undertake installing the Linux VM, if that's OK. The Linux testing is enabled on Travis-CI now, any PRs I submit will be checked there---at least nominally. (All tests besides coarray_navier_stokes
which is mysteriously failing with a SIGBUS bad memory access error...)
@finneyb is this problem persisting? I'm running into it unexpectedly and think it may be related to #28 / #161 and/or because the CMake logic is faulty and doesn't build the executable in some cases... I'm debugging now.
Also, there is a typo in https://github.com/sourceryinstitute/opencoarrays/commit/c7022fea43941ef579c988be88ae49ed5d1e5afb that causes the test executable to fail to build, but the test will still try to run...
On 12/6/2015 10:16 AM, afanfa wrote:
I added the suggested options at the appropriate place in install.sh and then executed install. Running ctest again results in the same error. The directory pde_solvers/navier-stokes is present, but the directory does not contain a file or directory named coarray_navier_stokes
22/22 Testing: coarray_navier_stokes 22/22 Test: coarray_navier_stokes Command: "/usr/local/bin/mpich/bin/mpiexec" "-np" "2" "/usr/local/bin/opencoarrays/opencoarrays-build/src/tests/integration/pde_solvers/navier-stokes/coarray_navier_stokes" Directory: /usr/local/bin/opencoarrays/opencoarrays-build "coarray_navier_stokes" start time: Dec 06 21:03 PST
Output:
[proxy:0:0@madison] HYDU_create_process (/usr/local/bin/mpich-3.1.4/src/pm/hydra/utils/launch/launch.c:75): execvp error on file /usr/local/bin/opencoarrays/opencoarrays-build/src/tests/integration/pde_solvers/navier-stokes/coarray_navier_stokes (No such file or directory) [proxy:0:0@madison] HYDU_create_process (/usr/local/bin/mpich-3.1.4/src/pm/hydra/utils/launch/launch.c:75): execvp error on file /usr/local/bin/opencoarrays/opencoarrays-build/src/tests/integration/pde_solvers/navier-stokes/coarray_navier_stokes (No such file or directory)
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = PID 26466 RUNNING AT madison = EXIT CODE: 255 = CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES