Consider to use TCP socket for inter process communication

[sile]

Currently, the standard input and output are used to communicate with external program problems and solvers but it turned out that it's not a good design. Because many Python libraries use the standard output as logging and debugging purposes, those are easily conflicting with the kurobako IPC. By using TCP instead, this issue could be resolved (but it would make the implementation more complicated).

[contramundum53]

How about using named pipes instead? It might be easier.

[sile]

Sounds interesting. However, AFAIK, there is no cross-platform named pipe API. So, I think, we need to limit the supported platform or provide fallback ways if named pipes are adopted as the IPC protocol.

[contramundum53]

In Unix the following Python code using anonymous pipes works.

import os
import subprocess

pipes = [os.pipe() for i in range(3)]
for r,w in pipes:
    os.set_inheritable(w, True)

subp = subprocess.Popen("python3 -c 'import sys, os; ws = [int(x) for x in sys.argv[1:]]; [print(f\"Hello{i}\\n\", file=os.fdopen(ws[i], \"w\")) for i in range(3)]' " + ' '.join([str(w) for r,w in pipes]), shell=True, close_fds=False)
subp.wait()
for r,w in pipes:
    r_fd = os.fdopen(r)
    print(r_fd.readline())

I don't have a Windows environment to test it, but according to the documentation this should also work in Windows.

[sile]

Thank you for sharing the code. If both runner and solver (or problem) process were written in Python, we could use the API. However, the language of the runner side is Rust, and AFAIK it (the standard library) doesn't provide the same cross-platform pipe functionality with Python. So we need to implement the feature (cross-platform pipe API compatible with Python) on our own (or find a crate implementing that). Of course, if you're interested in implementing that, it's more than welcomed and I can review the PR.

[contramundum53]

https://docs.rs/os_pipe/latest/os_pipe/ seems to be a cross-platform library for handling pipe creation. Also, it seems that in Rust all file descriptors are inherited by the child process.

[sile]

👍 > os_pipe