is it possible to access the same reg from 2 sessions? also destroys reg state completely

mllg / batchtools

Tools for computation on batch systems

https://mllg.github.io/batchtools/

GNU Lesser General Public License v3.0

172 stars 51 forks source link

is it possible to access the same reg from 2 sessions? also destroys reg state completely #136

Closed berndbischl closed 7 years ago

berndbischl commented 7 years ago

because:

that was was possible with BJ, and now very bad things seem to happen.

what we did was:

submit a large number of jobs from 1 session
while that was still going on, log into the cluster with another ssh
then simply call getStatus in the 2nd session

that produced (i think) also the bug here: #135

but later it was even worse: that bug #135 didnt happen. but we had 90% jobs finished. then that "done" state vanished from the status database. i can still see the result files on disk. but have apparently no chance to load or reduce them....

--> not good

berndbischl commented 7 years ago

to move furher:

1) it MUST be documented whether logging from 2 sessions is dangerous or not.

2) if the answer is "yes", and it seems to be, then:

a) it would be good if users can be prohibited from doing this. i know this is probably not that simple, but maybe at least think about this.

b) the main reason why people do this is because submitJobs can be a long operation, and it is blocking. i want to see the status of jobs in between, so i can react to errors, etc. a very easy fix would be to write the status into a file, that i can lookat in between. or display something in the current R session (that would be better for users, but harder to implement)

mllg commented 7 years ago

I'm working on this in the ro_mode branch.

Simultaneous access is dangerous, this is now clearly documented in loadRegistry()
Per default, you now load a registry in a read-only mode where you can do everything which does not change the state of the registry: monitoring status, reducing results, ...
If the modification time of the registry file on the file system is changed in the background, the registry will be set to read-only. This is rather heuristic and not effective for all scenarios, I'm still looking into it.
https://github.com/r-lib/filelock/ might be a much better alternative in the long run

mllg commented 7 years ago

This is now on master and CRAN. Let me know if you encounter any problems.