Extension to EXECUTE_COMMAND_LINE

[jacobwilliams]

It would be nice for some use cases if EXECUTE_COMMAND_LINE intrinsic had some kind of process ID optional output that could be used to check the status (and maybe also kill) a process executed asynchronously via wait=.false.

So something like this:

call execute_command_line('my_long_process.exe',wait=.false.,pid=pid)
do
  call cpu_time(t1)
  call do_some_stuff()
  if (process_still_running(pid)) then    ! a new intrinsic
    call cpu_time(t2)
    if (t2 - t1 >= max_time_to_wait) then
      call kill_process(pid)    ! a new intrinsic
      exit
    else
      t1 = t2
    end if
  else
    exit
  end if
end do

[marshallward]

It's been a long wish of mine to have some sort of standardized POSIX support within Fortran, either natively or through a public library. Something like that would go a long towards supporting native process control, along with a wide range of other resources.

[sblionel]

POSIX is dead. Intel Fortran supports a lot of POSIX routines, but it;'s hardly worth using them.

The problem I see with the proposal here is that it assumes host system support for this sort of concept. I implemented EXECUTE_COMMAND_LINE for Intel Fortran and it was complex enough across the various platforms to do the waiting properly. The Fortran standard tries not to tie itself down to implementation details. I might be able to see some sort of ID-based thing like asynchronous I/O, but requiring the ability to kill an asynchronous task seems a stretch.

[marshallward]

I'm not sure what you mean when you say that POSIX is dead, and perhaps it's not what I'm actually suggesting. All I meant was that I would like access to the lower level OS operations and resources like file handlers, process IDs, and system calls.

Greater access to these resources would enable development of things like greater subprocess control without requiring extensions of the language.

[sblionel]

The Fortran interfaces to POSIX were abandoned by the POSIX committee years ago, and are at the F77 level.

The Fortran language tries very hard to avoid baking in host environment aspects. Things such as "subprocess" are platform-specific. I know there is a general tendency to think that UNIX and Windows are the entire world, but they are not.

You already have access to lower-level APIs in most compilers, and you would probably end up spurning any standard interface for the more fine-grained access the OS offers in most cases.

[certik]

@sblionel I had exactly this discussion with a few colleagues recently. Besides HPC (mostly some flavor of Linux/UNIX), Linux, Windows, macOS and then platforms like Android, iOS --- all of which have a file system and processes (as far as I know) --- what are some other common platforms that Fortran users are running on? Is there a Fortran platform that does not have a filesystem, or does not have processes?

[sblionel]

It's not lack of a file system that's the issue - Fortran pretty much assumes there is one in how it defines OPEN/CLOSE/INQUIRE, though it avoids being specific. Process control varies a lot. In addition to Windows, MacOS and UNIX/Linux systems, Fortran is also used on CrayOS and OpenVMS. There may be other platforms that don't spring immediately to mind.

But the fundamental issue is that there is a strong desire to keep the standard away from specifying such low-level details as the language (and programs) have a long life and technologies change. For example, a few years ago we rejected a proposal to add a way to specify allocation from "fast memory", since we knew this was a short-term thing.

My personal opinion is that building low-level stuff into the language is a waste of effort better placed elsewhere. It's not as if you can't do all these things already with interfaces to system APIs, and my experience is that most applications want to go beyond what things built-in to the language could offer.

[jacobwilliams]

I don't see this as a low-level thing as far as a user of the language is concerned. We have the ability to start a process. It is logical that we should have to ability to check it and end it. Sure the internal details will vary for different platforms. Python certainly has the ability to kill processes, and I'm pretty sure that Python runs on as many or more platforms as Fortran (certainly more than Intel Fortran) so this is pretty much a solved problem as far as I'm concerned.

From my point of view, the current asynchronous option for execute_command_line is basically useless. I can't imagine a situation where I would want to start a process, and then just hope for the best that it finishes.

We can, of course, do it with interfaces to system APIs (this is what I do), but then we have to write a massive amount of code that is different for different platforms. I think it's better to make the compiler writers do that than the users. :)

[certik]

This seems like a perfect candidate for inclusion in #104.