Closed ahmet-uyar closed 4 years ago
@ahmet-uyar does this supported via command line? There is a restart command available.
@ahmet-uyar does this supported via command line? There is a restart command available.
@supunkamburugamuve Users submit the job as usual with the submit command. However, two configuration parameters need to be set up. Job id of the job to be restored (twister2.job.id) and the restore flag (twister2.checkpointing.restore.job).
@chathurawidanage we can use the restart command to this purpose? I think specifying these are configuration parameters can be hard for the user?
Yes. Restart command should work here.
./bin/twister2 restart
@chathurawidanage we can use the restart command to this purpose? I think specifying these are configuration parameters can be hard for the user?
@supunkamburugamuve let me look into that.
@supunkamburugamuve and @chathurawidanage
I implemented restart command as: $ twister2 restart $cluster-type jobID
I also implemented clear and clearall: $ twister2 clear $cluster-type jobID $ twister2 clearall $cluster-type
I think we can merge it now.
Ahemt could you please merge the latest changes from master to your branch? Seems merging is blocked due to conflicts.
@chathurawidanage and @supunkamburugamuve i merged msater branch and also did a few improvements. now it seems ok.
I implemented job restarts from checkpoints. Failed or killed checkpointed jobs can be restarted from persistent storage. Users shall provide the jobID of the previous job. They shall specify jobID as the value of the following parameter: twister2.job.id They shall also set the following config parameter to restore the job: twister2.checkpointing.restore.job
I have also thrown an exception when there is send/receive exception at TCPChannel. So that workers can fail and restart properly.