Consider adding the output of StartProcess to the operator

[bruno-f-cruz]

Currently, StartProcess ouputs the exit code of the process (e.g. 0). For some processes (e.g. running python scripts) would be useful to be able to have the output in the console available as a string in Bonsai instead of the exit code. This feature could be added as an optional parameter to the node.

Alternatively, a StartProcess(Python) that, similarly to VideoWriter(FFMPEG) launches a python sript and assumes that the user wants to have access to the console output. This could also present an opportunity to tap onto the stderror of the process and afford users a way to deal with such exceptions (e.g. crash the workflow).

[glopesdev]

Another alternative is to have a CreateProcess node that just prepares and creates the Process object and returns it. Then we could have downstream operators that can redirect / hook to stdin and stdout and eventually start the process.

[glopesdev]

There have also been reports that if the Bonsai parent process halts, the created child processes do not halt. Would be nice to double-check and test this.

[bruno-f-cruz]

I was revisiting this issue recently and I am still not sure what the behavior should be. I am going to leave this recipe here for future reference but this is still very unsatisfying. In my current use case, returning the full stdout all at once makes sense which greatly simplifies the observable logic (i.e. start the process and wait for it to cancel or return). However, even in this simpler scenario, I am still not sure what should be returned if the process is canceled. For now I guess that OperationCanceledException makes sense...

To generalize this pattern, we could keep reading lines from the stdout and emitting events as they are read. Once the process finishes, the sequence closes. Users can then choose to concatenate all the strings if the full stdout is required.

public override IObservable<string> Generate()
    {
        return Observable.StartAsync(cancellationToken =>
        {
            return Task.Factory.StartNew(() =>
            {
                using (var exitSignal = new ManualResetEvent(false))
                using (var process = new Process())
                {
                    process.StartInfo.FileName = FileName;
                    process.StartInfo.Arguments = Arguments;
                    process.StartInfo.UseShellExecute = false;
                    process.StartInfo.RedirectStandardError = true;
                    process.StartInfo.RedirectStandardOutput = true;
                    process.Exited += (sender, e) => exitSignal.Set();
                    process.EnableRaisingEvents = true;
                    process.Start();
                    using (var cancellation = cancellationToken.Register(() => exitSignal.Set()))
                    {
                        exitSignal.WaitOne();
                        if (!process.HasExited){
                            throw new TimeoutException("Process did not exit in time");
                        }
                        var stdError = process.StandardError.ReadToEnd();
                        if (!string.IsNullOrEmpty(stdError))
                        {
                            throw new InvalidOperationException(stdError);
                        }
                        return process.StandardOutput.ReadToEnd();
;
                    }
                }
            },
            cancellationToken,
            TaskCreationOptions.LongRunning,
            TaskScheduler.Default);
        });
    }