search

cooperative-computing-lab / makeflow-examples

Example workflows for the Makeflow workflow system.
32 stars 18 forks source link

Problem Running Makeflow with Shakespeare on SGE #22

Closed npotteig closed 6 years ago

npotteig commented 6 years ago

When I run the line makeflow -T sge -d all shakespeare.makeflow

I get an error when it runs into a problem with perl.sfx and the sge.wrapper with the error message repeated.

batch: qsub -cwd -j y -V -N 'perl.sfx' sge.wrapper Unable to read script file because of error: error opening file: No such file or directory

The makeflow will run if I use the Makeflow file instead of shakespeare.makeflow and example.makeflow will also run using sge so it is probably a problem with the shakespeare makeflow.

btovar commented 6 years ago

Where did you get the shakespeare.makeflow from?

Ben

On Tue, May 29, 2018 at 12:45 PM, npotteig notifications@github.com wrote:

When I run the line makeflow -T sge -d all shakespeare.makeflow

I get an error when it runs into a problem with perl.sfx and the sge.wrapper with the error message repeated.

batch: qsub -cwd -j y -V -N 'perl.sfx' sge.wrapper Unable to read script file because of error: error opening file: No such file or directory

The makeflow will run if I use the Makeflow file instead of shakespeare.makeflow and example.makeflow will also run using sge so it is probably a problem with the shakespeare makeflow.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cooperative-computing-lab/makeflow-examples/issues/22, or mute the thread https://github.com/notifications/unsubscribe-auth/AC8GYhX2k0UaNXHpkWHbBqyNHB8Q22Jjks5t3XsegaJpZM4UR2Uu .

npotteig commented 6 years ago

Ben,

I got shakespeare.makeflow from the github repository called makeflow-examples from the cooperative computing lab.

Nicholas

On Tue, May 29, 2018 at 2:59 PM, Benjamin Tovar notifications@github.com wrote:

Where did you get the shakespeare.makeflow from?

Ben

On Tue, May 29, 2018 at 12:45 PM, npotteig notifications@github.com wrote:

When I run the line makeflow -T sge -d all shakespeare.makeflow

I get an error when it runs into a problem with perl.sfx and the sge.wrapper with the error message repeated.

batch: qsub -cwd -j y -V -N 'perl.sfx' sge.wrapper Unable to read script file because of error: error opening file: No such file or directory

The makeflow will run if I use the Makeflow file instead of shakespeare.makeflow and example.makeflow will also run using sge so it is probably a problem with the shakespeare makeflow.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cooperative-computing-lab/ makeflow-examples/issues/22, or mute the thread https://github.com/notifications/unsubscribe-auth/ AC8GYhX2k0UaNXHpkWHbBqyNHB8Q22Jjks5t3XsegaJpZM4UR2Uu .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/cooperative-computing-lab/makeflow-examples/issues/22#issuecomment-392899565, or mute the thread https://github.com/notifications/unsubscribe-auth/AlyyTCTu-LLXI5P-ZuTIqEmAGBqumsEmks5t3Zp0gaJpZM4UR2Uu .

btovar commented 6 years ago

Of course, I forgot about that repo.

I was able to reproduce your error. Thanks for your report!

Ben

On Tue, May 29, 2018 at 4:14 PM, npotteig notifications@github.com wrote:

Ben,

I got shakespeare.makeflow from the github repository called makeflow-examples from the cooperative computing lab.

Nicholas

On Tue, May 29, 2018 at 2:59 PM, Benjamin Tovar notifications@github.com wrote:

Where did you get the shakespeare.makeflow from?

Ben

On Tue, May 29, 2018 at 12:45 PM, npotteig notifications@github.com wrote:

When I run the line makeflow -T sge -d all shakespeare.makeflow

I get an error when it runs into a problem with perl.sfx and the sge.wrapper with the error message repeated.

batch: qsub -cwd -j y -V -N 'perl.sfx' sge.wrapper Unable to read script file because of error: error opening file: No such file or directory

The makeflow will run if I use the Makeflow file instead of shakespeare.makeflow and example.makeflow will also run using sge so it is probably a problem with the shakespeare makeflow.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cooperative-computing-lab/ makeflow-examples/issues/22, or mute the thread https://github.com/notifications/unsubscribe-auth/ AC8GYhX2k0UaNXHpkWHbBqyNHB8Q22Jjks5t3XsegaJpZM4UR2Uu .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/cooperative-computing-lab/ makeflow-examples/issues/22#issuecomment-392899565, or mute the thread https://github.com/notifications/unsubscribe-auth/AlyyTCTu-LLXI5P- ZuTIqEmAGBqumsEmks5t3Zp0gaJpZM4UR2Uu .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cooperative-computing-lab/makeflow-examples/issues/22#issuecomment-392928310, or mute the thread https://github.com/notifications/unsubscribe-auth/AC8GYoDYI_UxO98Ley5U2DJwIa0fg3VRks5t3awcgaJpZM4UR2Uu .

tshaffe1 commented 6 years ago

I've at least partially tracked this down to a bug in our SGE submit code. It appears UGE here requires job name to be alphanumeric, so running a command starting with ./perl.sfx fails. I'm not convinced that this is the ultimate issue, though, as UGE's error is the opposite of helpful and I'm seeing file corruption. I wouldn't assume that qsub is that badly behaved.

tshaffe1 commented 6 years ago

Looks like qsub (at the CRC) was that badly behaved. This should be resolved by cooperative-computing-lab/cctools#1906

dthain commented 6 years ago

Ok to close?