search

aws / aws-parallelcluster

AWS ParallelCluster is an AWS supported Open Source cluster management tool to deploy and manage HPC clusters in the AWS cloud.
https://github.com/aws/aws-parallelcluster
Apache License 2.0
818 stars 309 forks source link

awsbsub ignores -w option when string command is provided #1059

Open mattwelborn opened 5 years ago

mattwelborn commented 5 years ago

Environment:

Bug description and how to reproduce: The -w flag is ignored by awsbsub when a string command is provided as an argument. Looking at cli/awsbatch/awsbsub.py, it can be seen that args.working_dir is only used in _compose_bash_command which is only called if args.command_file or not sys.stdin.isatty() or args.env.

Incorrect behavior example: Input: awsbsub -w /home/ec2-user pwd Output: 

2019-05-19T21:35:15+00:00: Job id: d519dc44-72e9-45d0-aae9-6eb5e502fafa
2019-05-19T21:35:15+00:00: Initializing the environment...
2019-05-19T21:35:15+00:00: Starting ssh agents...
2019-05-19T21:35:15+00:00: Agent pid 7
2019-05-19T21:35:15+00:00: Identity added: /root/.ssh/id_rsa (/root/.ssh/id_rsa)
2019-05-19T21:35:16+00:00: Mounting /home...
2019-05-19T21:35:16+00:00: Mounting shared file system...
2019-05-19T21:35:16+00:00: Starting the job...
2019-05-19T21:35:16+00:00: /

Correct behavior example: Input: echo pwd | awsbsub -w /home/ec2-user Output:

2019-05-19T21:35:18+00:00: Job id: 033c9be8-6964-4a67-86bd-eea069c55ba1
2019-05-19T21:35:18+00:00: Initializing the environment...
2019-05-19T21:35:18+00:00: Starting ssh agents...
2019-05-19T21:35:18+00:00: Agent pid 9
2019-05-19T21:35:18+00:00: Identity added: /root/.ssh/id_rsa (/root/.ssh/id_rsa)
2019-05-19T21:35:19+00:00: Mounting /home...
2019-05-19T21:35:19+00:00: Mounting shared file system...
2019-05-19T21:35:19+00:00: Starting the job...
2019-05-19T21:35:20+00:00: /home/ec2-user
lukeseawalker commented 5 years ago

Hi @mattwelborn, good catch. The parsing of the -w flag was not implemented when a string command is provided as an argument. This because in this case the command is sent directly to AWS Batch instead of composing a wrapper script (which is in charge to switch the working directory). I'm marking this as bug, in order to have the user experience consistent. Just for curiosity what's the use case of using a simple command provided as argument?

mattwelborn commented 5 years ago

I was using it in a bash loop in order to run a computationally intensive code on each of a few thousand input files.