search

fractal-analytics-platform / fractal-server

Fractal backend
https://fractal-analytics-platform.github.io/fractal-server/
BSD 3-Clause "New" or "Revised" License
10 stars 3 forks source link

Support creation of multiple SSH connections #1782

Closed tcompa closed 4 days ago

tcompa commented 1 week ago

Right now, we are defining a single FractalSSH object during the app startup. As soon as we support different SSH service users, we cannot have a single FractalSSH object any more, but we need one for each service users.

The current issue concerns supporting this new feature.

A future issue may concern the scalability of this solution in terms of maximum SSH connections (cause they would naively scale with the number of service users).

jluethi commented 1 week ago

Just for my high-level understanding: Do we need to maintain constant ssh connections or do we start them when users trigger an action (& potentially for some time after that)?

e.g. in a scenario with 100 service users, but only 10 active ones: Is there a downside to having the other 90 service users configured?

tcompa commented 1 week ago

Just for my high-level understanding: Do we need to maintain constant ssh connections or do we start them when users trigger an action (& potentially for some time after that)?

For the moment, fractal/paramiko does not impose any finite lifetime on SSH connections: connections are established the first time that one needs them, and then remain "forever" (although I cannot exclude that they will expire for reasons that we don't control).

Whether this is an issue depends on how the caveat above plays out: are these connections eventually closed from the remote side? And if yes, how often?

If the connection lifetime is long (in some units), then more work will be needed on the Fractal side. The idea is that we will have to setup a pool of connections that we control more dynamically, and that we would stop after some inactivity time. For the moment this is fully TBD.

A simpler solution (also fully TBD) would be to always deactivate the connection after every single command, but this would lead to overheads, and possibly to concurrent-execution conflicts (because connections are shared among different task-collections or job-execution activities).

(side remark: notice that the number of connection can be as large as the number of service users times the number of app workers)