oar-team / oar3

OAR: versatile resource and job manager for cluster (third generation)
Other
8 stars 11 forks source link

oarsh issue #7

Open nikosT opened 2 years ago

nikosT commented 2 years ago

I'm running the oar-docker-compose "cluster". After logging via CLI to frontend I submitted successfully an interactive job:

user1@frontend:/$ oarsub -I -l nodes=2
# INFO:  Moldable instance:  1  Estimated nb resources:  2  Walltime:  3600
OAR_JOB_ID= 2
Interactive mode: waiting...
Starting...
Connect to OAR job 2 via the node node1
user1@node1:~$  

Without closing this CLI, on another CLI, I logged in to the frontend again, then I checked that job is running, however, I get no access to any of the node1 and node2 machines via oarsh (and by setting the OAR_JOB_ID env. var.),

user1@frontend:/$ oarstat -f
Job id    S User     Duration          System message
--------- - -------- ----------------- ------------------------------------------------
2         R user1       0:05:22 R=2,J=I,Q=default                               
user1@frontend:/$ OAR_JOB_ID=2 oarsh node1
oarsh: Permission denied, seems like job 2 is not yours.

If I run oarsh inside any of the reserved nodes, then I get access, but not through the frontend. Additionally, the job as shown above belongs to the same user user1.

NODE List is shown below:

user1@node1:~$ cat $OAR_NODEFILE 
node1
node2
adfaure commented 2 years ago

Hello,

I can reproduce the error. I will look at it, in the meantime you can use oarsub -C <jobid> to connect to the job.

nikosT commented 2 years ago

Yes, I did and it works. I think the issue has to do with the fact that the way the oar-docker-compose is implemented, the containers do no share the same "file system". So, if a file is created in /home/user1 then it is visible only from the "machine" it was created and NOT all the containers.