22-120r2 Generics formal requirements: Type-bound procedures

[zjibben]

https://github.com/j3-fortran/generics/blob/4e273a9b394a0e48a217fcf307c0ddb598475dd2/J3-Papers/Generics_Requirements.txt#L214-L217

Does this mean subgroup is not considering type-bound procedures for 202y, or only that it'll be explored later and still planned for 202y?

And to be clear, this means it would not be possible to call type-bound procedures in any templated code?

[tclune]

Well, after Malcolm's comments last night, I expect this bullet to change. We consider using type-bound methods in this manner a way to create rather limited templates and it should be discouraged. But for "consistency with the language" and to help with gritty real-world cases, we probably have to allow it. Note that satisfying strong concepts means there will be lots of boiler plate when one uses this in a template.

[tclune]

(Also note - you really should look at the J3 paper to see what is said. Some things have changed.)

[zjibben]

We consider using type-bound methods in this manner a way to create rather limited templates and it should be discouraged.

It's possible I'm thinking of a different kind of use for type-bound procedures. For instance, one use-case is generic implementations of optimization algorithms. E.g. pass a templated function any type T which has a specific eval method:

template eval_tmpl(T)

  type :: T
  contains
    procedure :: eval
  end type
  interface
    real function eval(this, x)
      type(T), intent(inout) :: this
      real, intent(in) :: x
    end function
  end interface

contains

  real function find_root(func)
    type(T), intent(inout) :: func
    ...
    f0 = func%eval(x)
    ...
  end function find_root

end template

Wouldn't the strong-concepts boilerplate for the eval method be similar to the boilerplate for any other function?

[tclune]

Yes. And when it is only one type-bound procedure with one argument, it's not so bad. But it can snowball quickly. And if it is only one, you could with just about the same amount of code pass in an extra procedure as a parameter and use that and the procedure would then invoke the type-bound procedure. But Malcolm is right that it would be better for the template to do that rather than forcing all client code.

But consider the case of a type bound operator, say +. That would work with some derived types. And another template could be defined with an operator template argument. But you'd not be able to do both in one template. This is annoying.

[tclune]

@zjibben I think your functor example above is an excellent one, and will probably use it as an example when we mod the papers.

[zjibben]

Right, I can see this snowballing quickly, but even moreso if clients need to wrap all type-bound procedures in extra functions.

Can you describe your + example more? I'm not following.

I'm glad to help! This kind of use-case is an important one I feel; I'm happy it'll be useful.

[certik]

With the current paper which does not allow type bound procedures, how exactly would this be written:

template eval_tmpl(T)

  type, template :: T
  end type

  interface
    real function eval(this, x)
      type(T), intent(inout) :: this
      real, intent(in) :: x
    end function
  end interface

contains

  real function find_root(eval)
    type(eval) :: callback
    ...
    f0 = callback(x)
    ...
  end function find_root

end template

[zjibben]

Based on my understanding, there'd be two options.

First, the template would need to accept a function: template eval_tmpl(T, eval)

If eval is a module-private procedure in the module which defines our functor type, clients need to write a wrapper function. This is how I write most type-bound procedures; they are private at the module level, and public at the type-level. This can get onerous if there are lots of member functions:

use mytype_m
instantiate eval_tmpl(mytype, mytype_eval)

type(mytype) :: func

x0 = find_root(func)

contains

  real function mytype_eval(func, x)
    type(mytype), intent(inout) :: func
    real, intent(in) :: x
    mytype_eval = func%eval(x)
  end function

Or if eval is a module-public function in mytype_m, you could use that directly. In most code I write, this is not the case.

use mytype_m
instantiate eval_tmpl(mytype, eval)

type(mytype) :: func

x0 = find_root(func)

Now if templates can use type-bound procedures, eval can remain module-private (exposed only through the type interface) and we would write:

use mytype_m
instantiate eval_tmpl(mytype)

type(mytype) :: func

x0 = find_root(func)

PS: In retrospect, I should have used the name find_root_tmpl instead of eval_tmpl.

[everythingfunctional]

With the current paper which does not allow type bound procedures, how exactly would this be written:

Like this:

template eval_tmpl(T, eval)

  type, template :: T
  end type

  interface
    real function eval(this, x)
      type(T), intent(inout) :: this
      real, intent(in) :: x
    end function
  end interface

contains

  real function find_root(func)
    type(T) :: func
    ...
    f0 = eval(func, x)
    ...
  end function find_root

end template

You were pretty close actually.

[tclune]

@certik Close. You need another parameter for the template:

template find_root_tmpl(T, eval)

  type, template :: T
  end type

  interface
    real function eval(this, x)
      type(T), intent(inout) :: this
      real, intent(in) :: x
    end function
  end interface

contains

  real function find_root(functor)
    type(T) :: functor
    ...
    f0 = eval(functor, x)
    ...
  end function find_root

end template

The user would then write a function

real function eval_wrap(obj, x) result(y)
    type(my_functor), intent(in) :: obj
    real, intent(in) :: x
    y = obj%eval(x)
end function

And instantiate with

instantiate find_root_tmp(my_functor, eval_wrap)

[certik]

(I am going to reopen this issue, I think we have not reached a conclusion what to do about this.)

[certik]

The reasons/arguments against adding type-bound procedures in the proposal were:

• Java allows both type bound and external procedures which leads to duplicating code in the library, in order to support both (My note: I would like to see an example of this to understand this better)
• In C++ (which also allows both) the move is away from inheritance and more towards composability via templates/concepts; why adding a feature that is not encouraged to use?
• We should start small and add more features. If we add more features at the beginning, we cannot take them away.

My question:

Q: How do Rust, Haskell and Go do it? A: Here are detailed examples of such a functor: https://github.com/j3-fortran/generics/blob/4e273a9b394a0e48a217fcf307c0ddb598475dd2/theory/comparison/comparison.md, and as you can see all three languages allow type bound procedures.

I think we should write how the example in comparison.md would look like using the latest proposal.

[zjibben]

In C++ (which also allows both) the move is away from inheritance and more towards composability via templates/concepts; why adding a feature that is not encouraged to use?

I think inheritance and using member functions in templates are separate issues. Templates allow you to write generic procedures, which call type-bound procedures, without using inheritance.

The comparison you linked is an excellent example. Writing Stringify in Fortran templates would run into this issue.

[everythingfunctional]

I think we should write how the example in comparison.md would look like using the latest proposal.

Agreed. I think the Rust example is probably the easiest to grok and the most similar, so I'll go with that one.

trait Stringer {
    fn string(&self) -> &'static str;
}

fn stringify<T : Stringer>(s: Vec<T>) -> String {
    let mut ret = String::new();
    for x in s.iter() {
        ret.push_str(x.string());
    }
    ret
}

would be equivalent to

restriction stringable(T, to_string)
  type :: T; end type
  interface
    function to_string(x) result(string)
      type(T), intent(in) :: x
      character(len=:), allocatable :: string
    end function
  end interface
end restriction

template stringify_tmpl(T, to_string)
  requires stringable(T, to_string)
contains
  function stringify(s) result(string)
    type(T), intent(in) :: s(:)
    character(len=:), allocatable :: string

    integer :: i

    string = ""
    do i = 1, size(s)
      string = string // to_string(s(i))
    end do
  end function
end template

and then

struct MyT {
}

impl Stringer for MyT {
    fn string(&self) -> &'static str {
        "X"
    }
}

fn main() {
    let v = vec![MyT{}, MyT{}, MyT{}];
    println!("{}", stringify(v));
}

would be equivalent to

type :: my_t
end type

function to_string(x) result(string)
  type(my_t), intent(in) :: x
  character(len=:), allocatable :: string

  string = "X"
end function

instantiate stringify_tmpl(my_t, to_string)

type(my_t), allocatable :: v(:)

v = [my_t(), my_t(), my_t()]
print *, stringify(v)

[zjibben]

Is x.string() in the Rust example a type-bound procedure? It seems like Rust's member functions are listed outside struct MyT {}, but I don't know the language. If so, and we were to mimic that in Fortran, it might be:

restriction stringable(T)
  type :: T
  contains
    procedure :: string
  end type
  interface
    function string(x) result(s)
      class(T), intent(in) :: x
      character(len=:), allocatable :: s
    end function
  end interface
end restriction

template stringify_tmpl(T)
  requires stringable(T)
contains
  function stringify(s) result(string)
    type(T), intent(in) :: s(:)
    character(len=:), allocatable :: string

    integer :: i

    string = ""
    do i = 1, size(s)
      string = string // s(i).string()
    end do
  end function
end template

and

type :: my_t
contains
  procedure :: string
end type

function string(x) result(s)
  class(my_t), intent(in) :: x
  character(len=:), allocatable :: s
  s = "X"
end function

instantiate stringify_tmpl(my_t)

type(my_t), allocatable :: v(:)

v = [my_t(), my_t(), my_t()]
print *, stringify(v)

[everythingfunctional]

Is x.string() in the Rust example a type-bound procedure?

Yes. Rust has a feature that Fortran doesn't have. You can add new member functions (type-bound procedures) to existing structs (derived types). The impl keyword gives you a place to do that, and at the same time ask the compiler to ensure that you're conforming to the named interface (i.e. Stringer).

[everythingfunctional]

restriction stringable(T)
  type :: T
  contains
    procedure :: string
  end type
  interface
    function string(x) result(s)
      class(T), intent(in) :: x
      character(len=:), allocatable :: s
    end function
  end interface
end restriction

This is basically exactly how I would propose to enable this, but what happens if you didn't write my_t, and it is spelled to_string instead? I would propose to allow something like:

instantiate stringify_tmpl(my_t(string => to_string))

There's a lot more complexity to deal with than just that though, which is why we wanted to punt that feature to 202Z.

[zjibben]

Thanks for the clarification, this Rust feature is interesting.

what happens if you didn't write my_t, and it is spelled to_string instead?

Hmm yes, this is a problem to consider. Same for the find_root_tmpl earlier, if I was handed a derived type with a differently-spelled eval method, or perhaps one with a slightly different interface (e.g., optional arguments). I think if I were to choose where to draw the complexity line though, I'd advocate for drawing it at the "method rename" capability, rather than punting type-bound procedures entirely. Just my two cents 😄

[tclune]

While I agree that the suggestion by @everythingfunctional could in theory allow templates involving type bound procedures (and data components!) to be more general, it seems a very large step, and I would certainly oppose that for this first rollout. But I would want to let plenary see that potential so that they can decide. I don't want to mislead with examples showing how non useful templates involving type-bound procedures are.

[tclune]

For the Stringable template above, this is also a rather large departure from current subgroup plans. We had almost gotten to the point that RESTRICTION could be replaced with a parameterized, named abstract interface:

 ABSTRACT INTERFACE   FOO(T1, T2, ...)
...

I suppose it could still be spelled that way, but introducing type specifications (ish) at that stage is odd. In some ways I like it, but am feeling rushed which is bad.

[everythingfunctional]

I believe the restriction block was somewhat anticipatory of the desire for type-bound stuff, so I'm glad we went with it.

[rouson]

In C++ (which also allows both) the move is away from inheritance and more towards composability via templates/concepts; why adding a feature that is not encouraged to use?

I think inheritance and using member functions in templates are separate issues. Templates allow you to write generic procedures, which call type-bound procedures, without using inheritance.

@zjibben Inheritance is inherently linked to type-bound procedures by default because of the standard's requirement that the passed-object dummy argument be polymorphic. If one wants to use type-bound procedure foo as a template parameter, one can break this link by

1. Passing a wrapper for foo as the template parameter or
2. Giving foo the nopass attribute.

but approach 2 helps only if the compiler exploits such information. I've watched this movie and multiple sequels in the coarray world with public claims that coarrays are "not ready for prime time" despite numerous papers showing excellent performance across a range of applications running on 80-130K cores. I fear that I will now watch a spin-off series on templates. Many compiler teams probably won't even have the resources or motivation to exploit approach 2 because it it's unlikely to appear in enough code to matter.

We're making a mistake to appease one strong and admittedly influential opinion, which makes me wish I could oppose this proposal both on technical and social grounds, but I lean toward supporting it because not doing templates would be even worse, given the results of WG5's community survey.

[rouson]

Q: How do Rust, Haskell and Go do it? A: Here are detailed examples of such a functor: https://github.com/j3-fortran/generics/blob/4e273a9b394a0e48a217fcf307c0ddb598475dd2/theory/comparison/comparison.md, and as you can see all three languages allow type bound procedures.

@certik I value the wisdom of the committee in learning from the advantages and the pitfalls of other languages. One example that I often use is the committee's decision to disallow multiple inheritance. I hope we have the opportunity to do something compelling that improves on other languages' approaches.

[zjibben]

In C++ (which also allows both) the move is away from inheritance and more towards composability via templates/concepts; why adding a feature that is not encouraged to use?

I think inheritance and using member functions in templates are separate issues. Templates allow you to write generic procedures, which call type-bound procedures, without using inheritance.

@zjibben Inheritance is inherently linked to type-bound procedures by default because of the standard's requirement that the passed-object dummy argument be polymorphic.

@rouson Fair point, and I share your concern about compiler support. What I meant to get at is just that type-bound procedures are useful even without using runtime polymorphism patterns (e.g., the functor and Stringer examples). So moving away from inheritance patterns doesn't necessarily mean moving away from member functions, even if in Fortran type-bound procedures take a class argument.

[tclune]

@zjibben Agreed. Containers are probably a prime example of this in STL. One generally should not inherit from an STL container, but the STL containers still have methods. Some thought has gone into which operations should be methods (e.g., size(), at(), begin(), ..) and which should be procedures. Avoiding the use of methods would create a lot of namespace pollution.