search

nextflow-io / nextflow

A DSL for data-driven computational pipelines
http://nextflow.io
Apache License 2.0
2.72k stars 621 forks source link

Adding parentheses to directory structure when using bin #1549

Open avsastry opened 4 years ago

avsastry commented 4 years ago

Bug report

Expected behavior and actual behavior

I'm currently working within a Dropbox folder that is auto-named as "Dropbox (UCSD)". The presence of these parentheses seem to be properly accounted for in all cases, except when I'm trying to run a script from the "bin" folder. In the .command.run file, I see the following code:

nxf_container_env() {
cat << EOF 
export PATH="/home/anand/Dropbox (UCSD)/Projects/nextflow_test/bin:\$PATH"
EOF
}

It seems that the export call to use the "bin" directory doesn't backslash the parentheses and the spaces, resulting in a run error:

  /bin/bash: eval: line 0: syntax error near unexpected token `('
  /bin/bash: eval: line 0: `export PATH=/home/anand/Dropbox (UCSD)/Projects/nextflow_test/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'

Steps to reproduce the problem

I can attach a working version if necessary

Program output


Mar-31 13:26:47.561 [main] DEBUG nextflow.cli.Launcher - $> nextflow run main.nf
Mar-31 13:26:47.710 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 20.01.0
Mar-31 13:26:47.794 [main] INFO  nextflow.cli.CmdRun - Launching `main.nf` [romantic_kirch] - revision: 4a8194bef6
Mar-31 13:26:48.151 [main] DEBUG nextflow.Session - Session uuid: f9bd066a-4c03-468f-b137-0a79fcc07474
Mar-31 13:26:48.151 [main] DEBUG nextflow.Session - Run name: romantic_kirch
Mar-31 13:26:48.154 [main] DEBUG nextflow.Session - Executor pool size: 8
Mar-31 13:26:48.290 [main] DEBUG nextflow.cli.CmdRun -
  Version: 20.01.0 build 5264
  Created: 12-02-2020 10:14 UTC (02:14 PDT)
  System: Linux 4.15.0-91-generic
  Runtime: Groovy 2.5.8 on OpenJDK 64-Bit Server VM 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08
  Encoding: UTF-8 (UTF-8)
  Process: 12645@sbrg-sastry [127.0.1.1]
  CPUs: 8 - Mem: 31.3 GB (23.2 GB) - Swap: 31.9 GB (31.9 GB)
Mar-31 13:26:48.470 [main] DEBUG nextflow.file.FileHelper - Can't check if specified path is NFS (1): /home/anand/Dropbox (UCSD SBRG)/Projects/dev_testing/test (folder)/work
  ext2/ext3

Mar-31 13:26:48.470 [main] DEBUG nextflow.Session - Work-dir: /home/anand/Dropbox (UCSD SBRG)/Projects/dev_testing/test (folder)/work [null]
Mar-31 13:26:48.529 [main] DEBUG nextflow.Session - Observer factory: TowerFactory
Mar-31 13:26:48.532 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Mar-31 13:26:48.981 [main] DEBUG nextflow.Session - Session start invoked
Mar-31 13:26:49.653 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Mar-31 13:26:49.679 [main] DEBUG nextflow.Session - Workflow process names [dsl1]: use_bin_folder
Mar-31 13:26:49.741 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Mar-31 13:26:49.741 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Mar-31 13:26:49.747 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Mar-31 13:26:49.754 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=8; memory=31.3 GB; capacity=8; pollInterval=100ms; dumpInterval=5m
Mar-31 13:26:49.904 [main] DEBUG nextflow.processor.TaskProcessor - Creating operator > use_bin_folder -- maxForks: 8; blocking: false
Mar-31 13:26:49.955 [main] DEBUG nextflow.script.ScriptRunner - > Await termination 
Mar-31 13:26:49.957 [main] DEBUG nextflow.Session - Session await
Mar-31 13:26:50.235 [Task submitter] DEBUG nextflow.executor.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Mar-31 13:26:50.242 [Task submitter] INFO  nextflow.Session - [cf/dd5d60] Submitted process > use_bin_folder
Mar-31 13:26:50.332 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: use_bin_folder; status: COMPLETED; exit: 127; error: -; workDir: /home/anand/Dropbox (UCSD SBRG)/Projects/dev_testing/test (folder)/work/cf/dd5d60f054d580a87bd60f9081f050]
Mar-31 13:26:50.380 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'use_bin_folder'

Caused by:
  Process `use_bin_folder` terminated with an error exit status (127)

Command executed:

  ./run_test.sh

Command exit status:
  127

Command output:
  (empty)

Command error:
  .command.sh: line 2: ./run_test.sh: No such file or directory

Work dir:
  /home/anand/Dropbox (UCSD SBRG)/Projects/dev_testing/test (folder)/work/cf/dd5d60f054d580a87bd60f9081f050

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
Mar-31 13:26:50.391 [main] DEBUG nextflow.Session - Session await > all process finished
Mar-31 13:26:50.391 [Task monitor] DEBUG nextflow.Session - Session aborted -- Cause: Process `use_bin_folder` terminated with an error exit status (127)
Mar-31 13:26:50.486 [main] DEBUG nextflow.Session - Session await > all barriers passed
Mar-31 13:26:50.497 [main] DEBUG nextflow.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=0ms; failedDuration=54ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=1; peakCpus=1; peakMemory=0; ]
Mar-31 13:26:50.607 [main] DEBUG nextflow.CacheDB - Closing CacheDB done
Mar-31 13:26:50.620 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

### Environment 

* Nextflow version: 20.01.0.5264
* Java version: 1.8.0_242
* Operating system: Linux
* Bash version: 4.4.20

### Additional context
stale[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

pditommaso commented 4 years ago

keep open

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

hukai916 commented 2 months ago

Same bug for me. Any update on it?

bentsherman commented 2 months ago

Maybe the bin path should be escaped with Escape.path() in addition to Escape.variable(): https://github.com/nextflow-io/nextflow/blob/2fb5bc07f2ad1309c9743b8675bb8003892e3eb7/modules/nextflow/src/main/groovy/nextflow/processor/TaskProcessor.groovy#L2090-L2091

hukai916 commented 2 months ago

Thanks @bentsherman, these headaches are all caused by the stupid decision made by the Dropbox team to include spaces in folder names. The solution (link below) provided by rwmorris solves the headache. Hope it helps someone else.

https://www.dropboxforum.com/t5/Plans-and-Subscriptions/How-to-remove-spaces-from-Dropbox-folder-names-spaces-causing/td-p/191195

pditommaso commented 2 months ago

@bentsherman worth to give a try to it