search

saleyn / erlexec

Execute and control OS processes from Erlang/OTP
https://hexdocs.pm/erlexec/readme.html
Other
525 stars 139 forks source link

Establishing/Supervising Communications With a Long-Running, Data Streaming Python Process #146

Closed Maxximiliann closed 3 years ago

Maxximiliann commented 3 years ago

Ubuntu 20.04.1 LTS Erlang/OTP 23 [erts-11.1] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:1] [hipe] Elixir 1.10.3 (compiled with Erlang/OTP 22)

Our Elixir project requires initiating and then supervising communications with a long-running, data streaming Python process. Data will be pushed to its Elixir counterpart once every second. Both processes are running on the same machine.

Is your library ideally suited for this? If so, where can we find recipes for such a use case?

saleyn commented 3 years ago

If the amount of data sent to Elixir by that process is small, then yes. All I/O of OS child processes spawned by erlexec that is sent to Elixir is funneled through exec-port executable. Read the examples in the accompanying documentation.

Maxximiliann commented 3 years ago

Small? Where do the docs explicate such limitations?

saleyn commented 3 years ago

There is no hard limitation - you just need to try and see if the amount of data your process is producing doesn't cause too high CPU utilization in exec-port. So far no one observed that being an issue, but you haven't given any details about your use case here, so it's all speculative.