search

gablab / mindhive

scripts, wiki and issues related to mindhive
mindhive.mit.edu
7 stars 77 forks source link

pbs jobs stuck in 'Q' state #28

Closed dosher closed 12 years ago

dosher commented 12 years ago

The cluster seems to be barely trickling jobs through, keeping them in the queued state. Also and possibly related, pbsnodes is outputting strange unicode characters in its list of jobs.

dosher commented 12 years ago

also strange: right now there are 40 jobs in 'Q', visible through qstat. However, qstat -Q reports only 35 jobs!

$ qstat | grep "Q" | wc -l
40
qstat -Q | tail -n 3 | awk '{print $1, $6}'
recon 0
mindhive 35
bigmem 0

Perhaps the scheduler is queuing jobs and forgetting about them, keeping them stuck in 'Q'?