Drain is locked while reading an output of python (or other program) REPL

[tengomucho]

I tried to use the "drain" example to read the output. The idea is to have a program that could send commands to a python REPL and read/parse the output. The problem is that, even if I set the nonblocking option, it seems to get stuck. Note that I have seen the same problem on Linux and Windows. Note that this does not seem to happen if I write my own child process reading input and sending output. Any idea or hint on how to fix that? Here's the code to reproduce it:

#include <reproc++/drain.hpp>
#include <reproc++/reproc.hpp>

#include <array>
#include <chrono> // std::chrono::seconds
#include <iostream>
#include <thread> // std::this_thread::sleep_for

static int fail(std::error_code ec)
{
  std::cerr << ec.message();
  return ec.value();
}

// Uses `reproc::drain` to show the output of the given command.
int main(int argc, const char **argv)
{
   reproc::options options;
  options.nonblocking = true;
  reproc::process process;

  const char *cmd[] = { "python", NULL };
  std::error_code ec = process.start(cmd);

  if (ec == std::errc::no_such_file_or_directory) {
    std::cerr << "Program not found. Make sure it's available from the PATH.";
    return ec.value();
  } else if (ec) {
    return fail(ec);
  }

  std::string output;
  reproc::sink::string sink(output);

  // sleep a bit to let the REPL fire up...
  std::this_thread::sleep_for(std::chrono::milliseconds(200));
  uint8_t repl_cmd[] = "exit()\n";
  process.write(repl_cmd, sizeof(repl_cmd));

  ec = reproc::drain(process, sink, sink); // <- It's stuck in the read!
  if (ec) {
    return fail(ec);
  }

  std::cout << output << std::flush;

  int status = 0;
  std::tie(status, ec) = process.wait(reproc::infinite);
  if (ec) {
    return fail(ec);
  }

  return status;
}

[DaanDeMeyer]

Python needs stdin to be closed before it starts executing commands, just add process.close(reproc::stream::in); after you've finished writing all the input and it'll work.

[tengomucho]

I added process.close as you told me, and it exits after the write, but my output string is empty... Furthermore, what if I wanted to keep sending input to the process, could I do it later?

[DaanDeMeyer]

I guess this means sending a newline just isn't sufficient to make the repl interpret the written input from stdin. We would need the correct character sequence to make the python repl progress further to make this work. I'm a bit sceptical whether this is a reproc specific problem. If this is something generic and the solution can be applied to more than just python, I'd be happy to include some helper function to 'flush' the output to the subprocess so it gets interpreted. I checked and we do write all the input correctly to the pipe so that shouldn't be a problem I think.

[DaanDeMeyer]

Closing this one as it's pretty old, feel free to comment if you're still having issues with this