Open gdevenyi opened 7 years ago
Student of mine was submitting a job with 500000 (long) lines and "-c 4000"
When jobs started, they all failed with:
/var/spool/torque/mom_priv/jobs/43286355-1.gpc-sched-ib0.SC: xrealloc: cannot allocate 18446744072881863172 bytes (249856 bytes allocated)
A quick bit of googling indicates that bash pipes have limits and we were overflowing some buffer somewhere.
(testing right now) using individual jobs resolves this.
We should maybe provide some warning about very very large jobfiles.
Student of mine was submitting a job with 500000 (long) lines and "-c 4000"
When jobs started, they all failed with:
A quick bit of googling indicates that bash pipes have limits and we were overflowing some buffer somewhere.
(testing right now) using individual jobs resolves this.
We should maybe provide some warning about very very large jobfiles.