sourceryinstitute / OpenCoarrays

A parallel application binary interface for Fortran 2018 compilers.
http://www.opencoarrays.org
BSD 3-Clause "New" or "Revised" License
246 stars 56 forks source link

RFE: expand type support in opencoarrays module #454

Closed rouson closed 6 years ago

rouson commented 7 years ago
Avg response time
Issue Stats

Request for Enhancement (RFE)

As pointed out by Malcolm Cohen on the WG5 Fortran standards committee mailing list, most or all of the type-specific specific procedures in src/extensions/opencoarrays.F90 can be collapsed down to a set of procedures in which the first argument is declared unlimited polymorphic (class(*)). This will both simplify the code and broaden the supported use cases.

As an example, the following program demonstrates that this approach which works at least with GCC 5 and higher:

program main
  implicit none
  integer :: b(10,10)
  call print_storage_info(b)
contains
  subroutine print_storage_info(a)
    class(*), intent(in), target, contiguous :: a(..)
    print *,storage_size(a),size(a)
  end subroutine
end program

The storage_size and size intrinsics should be used to replace the c_sizeofcalls that currently necessitate the type-specific implementations in opencoarrays.F90.

rouson commented 7 years ago

I started looking into this. One issue is choosing a compiler with which to test it. It should be a compiler that triggers the CMake code that builds of the extensions tests -- any non-gfortran compiler or a gfortran compiler version < 5, but let's focus on non-gfortran compilers. In deciding which ones to test, there are a few use cases to consider:

  1. Extending the capability of an existing Fortran 2008 compiler such as a. Intel b. Cray
  2. Accessing Fortran 2015 features in a Fortran 2003 compiler such as a. PGI b. flang c. NAG d. IBM

Arguments in favor of testing with the compilers in category 1 above include ease of implementation and greater likelihood that existing users will want additional parallel features (such as collective subroutines) because they're already using some parallel features (such as coarrays). Arguments in favor of testing with category 2 compilers are that these vendors might be most interested in adopting OpenCoarrays. An additional argument in favor of compiler 2b above is that it is open source, which might offer the greatest possibility of influencing the project itself and also might provide an avenue to increasing the interest of 2a.

Another consideration is whether the compiler supports the Fortran 2008 intrinsic function storage_size and the Fortran 2015 C-interoperability features such as assumed rank. These features greatly simplify the code in the opencoarrays module. The requisite features include, for example, assumed rank. At this time, I believe the list of compilers that support these features includes: I. Intel II. Cray III. IBM IV. GNU I'm going to first check whether flang supports the mentioned 2008 and 2015 features. If not, I'll switch to Intel. I just submitted a flang installation issue that's currently blocking me from evaluating it.

rouson commented 7 years ago

I also submitted an issue on the spack package manager recommended by the flang README.

rouson commented 7 years ago

It doesn't appear there's going to be a fix in the foreseeable future for building flang on macOS using spack. Building flang with spack on Lubuntu Linux is also failing for me, which I'll research and/or report. In the interim, I've sent the code below to Portland Group's J3 representative. Any compiler that either can compile the code below or supports the Fortran 2015 feature of assumed-size arguments for c_sizeof will facilitate the enhancement pondered in this issue. Unfortunately, gfortran appears to be the only compiler that can support the code below and of course the code below isn't needed with gfortran! Arrgghh... The Cray and Intel compiler both die with internal compiler errors on the code below even thought they nominal support the necessary features. I'll submit bug reports and I think I'll proceed with rewriting src/extensions/opencoarrays.F90 anyway.

program main
  use iso_c_binding, only : c_sizeof,c_int32_t
  implicit none
  integer(c_int32_t) :: array(10,1)
  associate(expected=>c_sizeof(array),calculated=>c_sizeof_assumed_rank(array))
    if (expected==calculated) then
      print *,"Test passed"
    else
      print *,"Test failed: expected ",expected,", calculated",calculated
    end if
  end associate
contains
   pure function c_sizeof_assumed_rank(a) result(c_sizeof_a)
     use iso_c_binding, only : c_size_t,c_int32_t
     class(*), intent(in), target, contiguous :: a(..) ! Fortran 2015 assumed-rank dummy argument
     integer(c_size_t) :: c_sizeof_a
     integer, parameter :: bits_per_byte=storage_size(0_c_int32_t)/c_sizeof(0_c_int32_t)
     c_sizeof_a= storage_size(a)*size(a)/bits_per_byte  ! Fortran 2008 storage_size intrinsic function
   end function
end program
rouson commented 7 years ago

The code in my last comment works with the IBM compiler so at least gfortran is not the only compiler that can handle it.

rouson commented 7 years ago

Bug reports have been submitted on the Cray and Intel compilers. Now I'll move forward with revising the module while we wait for the non-GNU compilers to support the requisite features.

rouson commented 6 years ago

And now I've also reported a bug in the NAG Fortran compiler based on the code at the beginning of this thread. An internal compiler error occurs in the NAG compiler when an unlimited polymorphic dummy argument has the contiguous attribute. I also submitted a feature request to NAG for the assumed-rank dummy argument. I'm tempted to write rank-specific versions of the procedures in opencoarrays.F90 and copy all arguments to assure contiguity and then test the opencoarrays module with the NAG compiler.

rouson commented 6 years ago

Intel reports that a fix for the bug related to this issue will be in their 18.0 Update 2 release.

zbeekman commented 6 years ago

@rouson I vaguely recall discussing marking this as won't fix and closing. What's the status of this? Are we still interested in implementing this? If not, let's mark as won't fix and close.

rouson commented 6 years ago

@zbeekman Actually, I might finally have the just right motivation for working on this. Morfeus-FD must compile with ifort and gfortran. The ifort requirement mostly restricts us to Fortran 2008, but the one part of Fortran 2018 that Intel has started to support is the TS 29113 Further Interoperability with C features and those are precisely the features needed for the changes that this issue contemplates. I'll have to check whether Intel has fixed the bug I reported to them. If they have, then it opens up the possibility for Morfeus-FD to use some Fortran 2018 parallel features through the opencoarrays module. Let's keep this open.

rouson commented 6 years ago

Good news. I submitted a feature request to Intel based on the code in the initial comment in this thread. Although that feature isn't yet supported, Intel reports that they have fixed the bug report that I also submitted based on the [workaround] in my second code example in this thread. I'll download the latest Intel Fortran compiler release and try the workaround.

https://github.com/sourceryinstitute/OpenCoarrays/issues/454#issuecomment-337765070

rouson commented 6 years ago

Arrgghh... first of all, the release isn't out yet. Second of all, even though Intel fixed the problem with the somewhat reduced code snippet that I submitted, an Intel engineer has confirmed that the compiler still can't handle the code in issue comment 337765070. I still plan to work on this once Intel's compiler handle either the code in the original comment in this thread or the workaround in the later comment.

rouson commented 6 years ago

Based on project needs, it appears this enhancement will be most useful with the Intel compiler in order to facilitate the use of OpenCoarrays collective subroutines with non-GCC compilers. This issue will be addressed after Intel finishes fixing related bugs. Currently, the required features only work with the Cray compiler (with the -Oipa0 flag) and with gfortran.

rouson commented 6 years ago

Intel Fortran Update 2 (Version 18.0.2.199 Build 20180210) now correctly compiles both the code block in the original post on this issue and the more complete example posted on October 18, 2017, in this thread with one minor workaround to switch a constant declaration to an associate block:

program main
  use iso_c_binding, only : c_sizeof,c_int32_t
  implicit none
  integer(c_int32_t) :: array(10,1)
  associate(expected=>c_sizeof(array),calculated=>c_sizeof_assumed_rank(array))
    if (expected==calculated) then
      print *,"Test passed"
    else
      print *,"Test failed: expected ",expected,", calculated",calculated
    end if
  end associate
contains
  pure function c_sizeof_assumed_rank(a) result(c_sizeof_a)
    use iso_c_binding, only : c_size_t,c_int32_t
    class(*), intent(in), target, contiguous :: a(..) ! Fortran 2018 assumed-rank dummy argument
    integer(c_size_t) :: c_sizeof_a
    associate(bits_per_byte=>storage_size(0_c_int32_t)/c_sizeof(0_c_int32_t))
      c_sizeof_a= storage_size(a)*size(a)/bits_per_byte ! Fortran 2008 storage_size intrinsic function
    end associate
  end function
end program

which is a workaround for Intel's compiler not supporting the Fortran 2018 feature that allows the c_sizeof intrinsic function to accept assumed-rank arguments.

With this latest workaround, Intel finally becomes the first compiler for which it is both feasible and useful to work on this issue. Doing so will allow code compiled with a Fortran 2008 compiler to access some Fortran 2018 features (e.g., collective subroutines) via the opencorrays module (in src/extensions/opencoarrays.F90), which justifies my having just added this issue to the low-hanging fruit project. The one challenging aspect of the project, however, will be adjusting the opencoarrays module to use the latest gfortran descriptors (which have to be constructed for passing arguments that OpenCoarrays will recognize even when compiled with a different compiler.

rouson commented 6 years ago

@zbeekman I marked this as low-hanging fruit. How do I make it appear in the To Do column on the low-hanging fruit project kansan? And what does "awaiting triage" mean?

zbeekman commented 6 years ago

When you mark an issue as part of a project from the issue's main page (i.e., here https://github.com/sourceryinstitute/OpenCoarrays/issues/454) it gets filed into the awaiting triage section on the project page. You must then find it in the "awaiting triage" section and drag it to the "To Do" column.

zbeekman commented 6 years ago

@rouson There are automation settings for the columns; I updated them so that issues that are moved to the low-hanging-fruit project will automatically end up in the To Do column. This should help make things more tractable.

rouson commented 6 years ago

This can finally be closed. A minimal example of this strategy has been implemented on the new armflang branch and testing will continue from there.