search

sourceryinstitute / OpenCoarrays

A parallel application binary interface for Fortran 2018 compilers.
http://www.opencoarrays.org
BSD 3-Clause "New" or "Revised" License
244 stars 58 forks source link

test failures with GCC7, but all pass with GCC8 #581

Closed mexas closed 5 years ago

mexas commented 6 years ago 
20474701626e> uname -a
FreeBSD 20474701626e.anet.bris.ac.uk 11.2-RELEASE-p2 FreeBSD 11.2-RELEASE-p2 #0: Tue Aug 14 21:45:40 UTC 2018     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64

FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads

20474701626e> cmake --version
cmake version 3.11.4

20474701626e> cc --version
FreeBSD clang version 6.0.0 (tags/RELEASE_600/final 326565) (based on LLVM 6.0.0)

For Fortran we use GCC. I noticed that using gfortran8 (for both MPICH and opencoarrays) makes all tests pass, but using gfortran7 gives multiple failures:

===>  Testing for opencoarrays-2.2.0_1
(cd /usr/ports/lang/opencoarrays/work/.build && ctest)
Test project /usr/ports/lang/opencoarrays/work/.build
Cannot create directory /usr/ports/lang/opencoarrays/work/.build/Testing/Temporary
Cannot create log file: LastTest.log
      Start  1: initialize_mpi
 1/75 Test  #1: initialize_mpi .........................   Passed    0.09 sec
      Start  2: register
 2/75 Test  #2: register ...............................   Passed    0.05 sec
      Start  3: register_vector
 3/75 Test  #3: register_vector ........................   Passed    0.05 sec
      Start  4: register_alloc_vector
 4/75 Test  #4: register_alloc_vector ..................   Passed    0.03 sec
      Start  5: allocate_as_barrier
 5/75 Test  #5: allocate_as_barrier ....................   Passed    1.03 sec
      Start  6: allocate_as_barrier_proc
 6/75 Test  #6: allocate_as_barrier_proc ...............   Passed    1.40 sec
      Start  7: register_alloc_comp_1
 7/75 Test  #7: register_alloc_comp_1 ..................   Passed    0.03 sec
      Start  8: register_alloc_comp_2
 8/75 Test  #8: register_alloc_comp_2 ..................   Passed    0.03 sec
      Start  9: register_alloc_comp_3
 9/75 Test  #9: register_alloc_comp_3 ..................   Passed    0.03 sec
      Start 10: async_comp_alloc
10/75 Test #10: async_comp_alloc .......................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.18 sec
      Start 11: async_comp_alloc_2
11/75 Test #11: async_comp_alloc_2 .....................   Passed    0.03 sec
      Start 12: comp_allocated_1
12/75 Test #12: comp_allocated_1 .......................   Passed    0.03 sec
      Start 13: comp_allocated_2
13/75 Test #13: comp_allocated_2 .......................   Passed    0.03 sec
      Start 14: alloc_comp_get_convert_nums
14/75 Test #14: alloc_comp_get_convert_nums ............***Failed  Required regular expression not found.Regex=[Test passed.
]  0.07 sec
      Start 15: send_convert_nums
15/75 Test #15: send_convert_nums ......................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.08 sec
      Start 16: sendget_convert_nums
16/75 Test #16: sendget_convert_nums ...................***Failed  Required regular expression not found.Regex=[Test passed.
] 98.58 sec
      Start 17: sendget_convert_char_array
17/75 Test #17: sendget_convert_char_array .............   Passed    0.05 sec
      Start 18: send_convert_char_array
18/75 Test #18: send_convert_char_array ................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.09 sec
      Start 19: alloc_comp_send_convert_nums
19/75 Test #19: alloc_comp_send_convert_nums ...........   Passed    0.03 sec
      Start 20: get_array
20/75 Test #20: get_array ..............................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.12 sec
      Start 21: get_self
21/75 Test #21: get_self ...............................   Passed    0.03 sec
      Start 22: get_convert_nums
22/75 Test #22: get_convert_nums .......................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.09 sec
      Start 23: get_convert_char_array
23/75 Test #23: get_convert_char_array .................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.09 sec
      Start 24: get_with_offset_1d
24/75 Test #24: get_with_offset_1d .....................   Passed    0.03 sec
      Start 25: whole_get_array
25/75 Test #25: whole_get_array ........................   Passed    0.03 sec
      Start 26: strided_get
26/75 Test #26: strided_get ............................   Passed    0.03 sec
      Start 27: send_array
27/75 Test #27: send_array .............................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.08 sec
      Start 28: convert-before-put
28/75 Test #28: convert-before-put .....................   Passed    0.03 sec
      Start 29: send_with_vector_index
29/75 Test #29: send_with_vector_index .................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.08 sec
      Start 30: strided_sendget
30/75 Test #30: strided_sendget ........................   Passed    0.03 sec
      Start 31: get_with_vector_index
31/75 Test #31: get_with_vector_index ..................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.13 sec
      Start 32: co_sum
32/75 Test #32: co_sum .................................   Passed    0.04 sec
      Start 33: co_broadcast
33/75 Test #33: co_broadcast ...........................   Passed    0.04 sec
      Start 34: co_min
34/75 Test #34: co_min .................................   Passed    0.04 sec
      Start 35: co_max
35/75 Test #35: co_max .................................   Passed    0.04 sec
      Start 36: syncall
36/75 Test #36: syncall ................................   Passed    1.45 sec
      Start 37: syncimages
37/75 Test #37: syncimages .............................   Passed    0.53 sec
      Start 38: syncimages2
38/75 Test #38: syncimages2 ............................   Passed    0.30 sec
      Start 39: duplicate_syncimages
39/75 Test #39: duplicate_syncimages ...................   Passed    0.23 sec
      Start 40: co_reduce
40/75 Test #40: co_reduce ..............................   Passed    0.04 sec
      Start 41: co_reduce_res_im
41/75 Test #41: co_reduce_res_im .......................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.10 sec
      Start 42: co_reduce_string
42/75 Test #42: co_reduce_string .......................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.12 sec
      Start 43: syncimages_status
43/75 Test #43: syncimages_status ......................   Passed    7.20 sec
      Start 44: sync_ring_abort_np3
44/75 Test #44: sync_ring_abort_np3 ....................   Passed    0.03 sec
      Start 45: sync_ring_abort_np7
45/75 Test #45: sync_ring_abort_np7 ....................   Passed    0.20 sec
      Start 46: simpleatomics
46/75 Test #46: simpleatomics ..........................   Passed    0.32 sec
      Start 47: hello_multiverse
47/75 Test #47: hello_multiverse .......................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.08 sec
      Start 48: coarray_burgers_pde
48/75 Test #48: coarray_burgers_pde ....................   Passed    0.04 sec
      Start 49: co_heat
49/75 Test #49: co_heat ................................   Passed    0.10 sec
      Start 50: asynchronous_hello_world
50/75 Test #50: asynchronous_hello_world ...............***Failed  Required regular expression not found.Regex=[Test passed.
]  0.09 sec
      Start 51: source-alloc-no-sync
51/75 Test #51: source-alloc-no-sync ...................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.32 sec
      Start 52: put-allocatable-coarray-comp
52/75 Test #52: put-allocatable-coarray-comp ...........   Passed    0.03 sec
      Start 53: get-put-allocatable-comp
53/75 Test #53: get-put-allocatable-comp ...............   Passed    0.03 sec
      Start 54: allocatable_p2p_event_post
54/75 Test #54: allocatable_p2p_event_post .............   Passed    0.04 sec
      Start 55: static_event_post_issue_293
55/75 Test #55: static_event_post_issue_293 ............   Passed    0.03 sec
      Start 56: co_reduce-factorial
56/75 Test #56: co_reduce-factorial ....................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.09 sec
      Start 57: co_reduce-factorial-int8
57/75 Test #57: co_reduce-factorial-int8 ...............***Failed  Required regular expression not found.Regex=[Test passed.
]  0.09 sec
      Start 58: co_reduce-factorial-int64
58/75 Test #58: co_reduce-factorial-int64 ..............***Failed  Required regular expression not found.Regex=[Test passed.
]  0.10 sec
      Start 59: issue-493-coindex-slice
59/75 Test #59: issue-493-coindex-slice ................***Failed  Required regular expression not found.Regex=[Test passed.
]  0.44 sec
      Start 60: issue-488-multi-dim-cobounds-true
60/75 Test #60: issue-488-multi-dim-cobounds-true ......***Failed  Required regular expression not found.Regex=[Test passed.
]  0.33 sec
      Start 61: issue-488-multi-dim-cobounds-false
61/75 Test #61: issue-488-multi-dim-cobounds-false .....***Failed  Required regular expression not found.Regex=[Test passed.
]  0.21 sec
      Start 62: issue-503-multidim-array-broadcast
62/75 Test #62: issue-503-multidim-array-broadcast .....***Failed  Required regular expression not found.Regex=[Test passed.
] 97.16 sec
      Start 63: issue-503-non-contig-red-ndarray
63/75 Test #63: issue-503-non-contig-red-ndarray .......***Failed  Required regular expression not found.Regex=[Test passed.
] 99.85 sec
      Start 64: issue-552-send_by_ref-singleton
64/75 Test #64: issue-552-send_by_ref-singleton ........***Failed  Required regular expression not found.Regex=[Test passed.
]  0.07 sec
      Start 65: issue-515-mimic-mpi-gatherv
65/75 Test #65: issue-515-mimic-mpi-gatherv ............***Failed  Required regular expression not found.Regex=[Test passed.
] 99.91 sec
      Start 66: image_status_test_1
66/75 Test #66: image_status_test_1 ....................   Passed    0.03 sec
      Start 67: image_fail_test_1
67/75 Test #67: image_fail_test_1 ......................   Passed    0.03 sec
      Start 68: image_fail_and_sync_test_1
68/75 Test #68: image_fail_and_sync_test_1 .............   Passed    0.03 sec
      Start 69: image_fail_and_sync_test_2
69/75 Test #69: image_fail_and_sync_test_2 .............   Passed    0.03 sec
      Start 70: image_fail_and_sync_test_3
70/75 Test #70: image_fail_and_sync_test_3 .............   Passed    0.03 sec
      Start 71: image_fail_and_status_test_1
71/75 Test #71: image_fail_and_status_test_1 ...........   Passed    0.03 sec
      Start 72: image_fail_and_failed_images_test_1
72/75 Test #72: image_fail_and_failed_images_test_1 ....   Passed    5.03 sec
      Start 73: image_fail_and_stopped_images_test_1
73/75 Test #73: image_fail_and_stopped_images_test_1 ...   Passed    0.03 sec
      Start 74: image_fail_and_get_test_1
74/75 Test #74: image_fail_and_get_test_1 ..............   Passed    0.04 sec
      Start 75: test-installation-scripts.sh
75/75 Test #75: test-installation-scripts.sh ...........   Passed    0.34 sec

65% tests passed, 26 tests failed out of 75

Total Test time (real) = 418.06 sec

The following tests FAILED:
Cannot create directory /usr/ports/lang/opencoarrays/work/.build/Testing/Temporary
Cannot create log file: LastTestsFailed.log
         10 - async_comp_alloc (Failed)
         14 - alloc_comp_get_convert_nums (Failed)
         15 - send_convert_nums (Failed)
         16 - sendget_convert_nums (Failed)
         18 - send_convert_char_array (Failed)
         20 - get_array (Failed)
         22 - get_convert_nums (Failed)
         23 - get_convert_char_array (Failed)
         27 - send_array (Failed)
         29 - send_with_vector_index (Failed)
         31 - get_with_vector_index (Failed)
         41 - co_reduce_res_im (Failed)
         42 - co_reduce_string (Failed)
         47 - hello_multiverse (Failed)
         50 - asynchronous_hello_world (Failed)
         51 - source-alloc-no-sync (Failed)
         56 - co_reduce-factorial (Failed)
         57 - co_reduce-factorial-int8 (Failed)
         58 - co_reduce-factorial-int64 (Failed)
         59 - issue-493-coindex-slice (Failed)
         60 - issue-488-multi-dim-cobounds-true (Failed)
         61 - issue-488-multi-dim-cobounds-false (Failed)
         62 - issue-503-multidim-array-broadcast (Failed)
         63 - issue-503-non-contig-red-ndarray (Failed)
         64 - issue-552-send_by_ref-singleton (Failed)
         65 - issue-515-mimic-mpi-gatherv (Failed)
Errors while running CTest
*** Error code 8

Stop.
make: stopped in /usr/ports/lang/opencoarrays
20474701626e> exit
exit
zbeekman commented 6 years ago

@mexas thanks for the report. I suspect something fishy might be happening in your environment to trigger so many failures with GFortran 7.3. One or two of those would be expected, but not 26...

The first step, is PLEASE, please, please set CTEST_OUTPUT_ON_FAILURE=ON in your environment, so that we can see the actual test output?

Second, I need to know specifically what version of GFortran you are using, and which version (vendor & version number) of MPI you are using.

I suspect that with GFortran 7.x there is a problem with either your MPI setup, or OpenCoarrays somehow getting linked against the wrong GFortran ABI or something similar. Did the build directory get completely wiped between building for GFortran 8 and GFortran 7?

mexas commented 6 years ago 
20474701626e> gfortran7 -v
Using built-in specs.
COLLECT_GCC=gfortran7
COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc7/gcc/x86_64-portbld-freebsd11.1/7.3.0/lto-wrapper
Target: x86_64-portbld-freebsd11.1
Configured with: /wrkdirs/usr/ports/lang/gcc7/work/gcc-7.3.0/configure --with-build-config=bootstrap-debug --disable-nls --enable-gnu-indirect-function --libdir=/usr/local/lib/gcc7 --libexecdir=/usr/local/libexec/gcc7 --program-suffix=7 --with-as=/usr/local/bin/as --with-gmp=/usr/local --with-gxx-include-dir=/usr/local/lib/gcc7/include/c++/ --with-ld=/usr/local/bin/ld --with-pkgversion='FreeBSD Ports Collection' --with-system-zlib --enable-languages=c,c++,objc,fortran --prefix=/usr/local --localstatedir=/var --mandir=/usr/local/man --infodir=/usr/local/info/gcc7 --build=x86_64-portbld-freebsd11.1
Thread model: posix
gcc version 7.3.0 (FreeBSD Ports Collection) 

20474701626e> mpif90 -v
mpifort for MPICH version 3.2.1
Using built-in specs.
COLLECT_GCC=gfortran7
COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc7/gcc/x86_64-portbld-freebsd11.1/7.3.0/lto-wrapper
Target: x86_64-portbld-freebsd11.1
Configured with: /wrkdirs/usr/ports/lang/gcc7/work/gcc-7.3.0/configure --with-build-config=bootstrap-debug --disable-nls --enable-gnu-indirect-function --libdir=/usr/local/lib/gcc7 --libexecdir=/usr/local/libexec/gcc7 --program-suffix=7 --with-as=/usr/local/bin/as --with-gmp=/usr/local --with-gxx-include-dir=/usr/local/lib/gcc7/include/c++/ --with-ld=/usr/local/bin/ld --with-pkgversion='FreeBSD Ports Collection' --with-system-zlib --enable-languages=c,c++,objc,fortran --prefix=/usr/local --localstatedir=/var --mandir=/usr/local/man --infodir=/usr/local/info/gcc7 --build=x86_64-portbld-freebsd11.1
Thread model: posix
gcc version 7.3.0 (FreeBSD Ports Collection)

If I remove (comment) /etc/make.conf, I don't get any failures with gfortran7. If I add these FFLAGS:

20474701626e> cat /etc/make.conf 
#DEVELOPER=yes
FFLAGS+= -O2 -pipe -march=bdver2 -mtune=bdver2
FFLAGS+= -funroll-loops --param max-unroll-times=4 -ftree-vectorize
FFLAGS+= -g
#DEFAULT_VERSIONS=gcc=8

I now get 2 failures with gfortran7. I added CTEST_OUTPUT_ON_FAILURE=ON. Here's the full make test log:

https://seis.bristol.ac.uk/~mexas/opencoarrays_gcc7_with_extra_FFLAGS.log

Yes, the build directory is fully wiped between builds (make clean).

zbeekman commented 6 years ago

Yes, the build directory is fully wiped between builds (make clean).

make clean will not wipe some sticky CMake configuration parameters. The best thing to do is an

rm -rf /path/to/build/dir # be careful that you don't nuke something important here!

but a make clean followed by a rm CMakeCache.txt should achieve the same results.

I'll try to look at the failures later today.

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

zbeekman commented 5 years ago

Failure logs are no longer available. Please let me know if you hit any issues with the latest OpenCoarrays releases. I will close this for now.