Closed scott-5 closed 1 month ago
@scott-5 It may be strange that the memory of the master node is not enough to maintain the program running
. Usually the dpgen process and the dpdispatcher process consume only a little memory. (And if OOM, I would rather 1. check the implementation of this software. 2. split the tasks into pieces. 3.Use better login in node)
I think @thangckt has supported this feature in #475.
@scott-5 It may be strange that
the memory of the master node is not enough to maintain the program running
. Usually the dpgen process and the dpdispatcher process consume only a little memory. (And if OOM, I would rather 1. check the implementation of this software. 2. split the tasks into pieces. 3.Use better login in node) Thanks for your advises. I'll check again.
I think @thangckt has supported this feature in #475.
It seems feasible. I'll close this issue before meeting anything unexpected.
Can the DPDispatcher program implement jump connection? If not, that is a good feature.
Some possible application scenarios : a program runs on a cluster(like dpgen), and then only the master node can connect to other resources, but the memory of the master node is not enough to maintain the program running. If the program is submitted to the computing node, but because the computing node is generally unable to connect to the network, it is impossible to distribute tasks to other resources from the computing node.
Therefore, if the master node can be used as a stepping stone for the computing node, the problem will be solved.