search

SIMEXP / psom

pipeline system for octave and matlab
http://psom.simexp-lab.org
Other
24 stars 14 forks source link

psom console on slurm #128

Closed poquirion closed 6 years ago

poquirion commented 6 years ago

Two things:

1 - It seems that singularity mode for the demon is broken on singularity/slurm/cedar:

poq@cdr544:~/rrg-pbellec/poq/project/CCNA$ cat /project/6003287/poq/project/CCNA/results/logs/deamon/deamon.eqsub
bash: cannot set terminal process group (23046): Inappropriate ioctl for device
bash: no job control in this shell
Traceback (most recent call last):
  File "/usr/local/bin/psom_worker.py", line 136, in <module>
    main()
  File "/usr/local/bin/psom_worker.py", line 128, in main
    w.start()
  File "/usr/local/bin/psom_worker.py", line 50, in start
    self.fp_out = open(self.std_out, 'w')
IOError: [Errno 2] No such file or directory: '/project/6003287/poq/project/CCNA/results/logs/worker/psomdeamon/worker.out'

2- The monitor is running the cpu too 100%, it's probably a loop that needs to be slowed down.

poquirion commented 6 years ago

Here is the script (in /tmp) that slurm send for execution

$ cat /tmp/psom_worker_qsub.Ltf9Up
#!/bin/bash
singularity exec -B /project -B /scratch -B /localscratch /home/poq/NIAK/niak-cog-1.1.3.img bash -ilc "psom_worker.py -d /project/6003287/poq/project/CCNA/results -w deamon"
poquirion commented 6 years ago

Ah, the daemon is not event configured to be in singulatity mode. Only workers, the garbadge collector and the manager!

Might add that in the future, closing for now