search

carpentries-incubator / workflows-nextflow

Workflow management with Nextflow and nf-core
https://carpentries-incubator.github.io/workflows-nextflow/
Other
19 stars 33 forks source link

Command error: ln: failed to create symbolic link : Operation not supported #153

Open unugwa opened 1 month ago

unugwa commented 1 month ago

I am currently going through the course and have downloaded the conda environment yaml. P.S. I had to update the Java version to 11 to get to work so it might be worth checking out.

At the moment, I am trying to complete episode 1 and run my first script. However, I keep getting 

Command error:
  ln: failed to create symbolic link 'ref1_1.fq.gz': Operation not supported

I can create symlinks. I can also run the commands successfully outside the workflow. Any guidance on how to resolve this, please?

ggrimes commented 1 month ago

can you upload your .nextflow.log file ?

unugwa commented 1 month ago

Here's it:

Jul-18 11:07:23.851 [main] DEBUG nextflow.cli.Launcher - $> nextflow run scripts/introduction/word_count.nf
Jul-18 11:07:23.920 [main] DEBUG nextflow.cli.CmdRun - N E X T F L O W  ~  version 24.04.3
Jul-18 11:07:23.936 [main] DEBUG nextflow.plugin.PluginsFacade - Setting up plugin manager > mode=prod; embedded=false; plugins-dir=/home/s2562160/.nextflow/plugins; core-plugins: nf-amazon@2.5.2,nf-azure@1.6.0,nf-cloudcache@0.4.1,nf-codecommit@0.2.0,nf-console@1.1.3,nf-ga4gh@1.3.0,nf-google@1.13.2-patch1,nf-tower@1.9.1,nf-wave@1.4.2
Jul-18 11:07:23.958 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Enabled plugins: []
Jul-18 11:07:23.959 [main] INFO  o.pf4j.DefaultPluginStatusProvider - Disabled plugins: []
Jul-18 11:07:23.961 [main] INFO  org.pf4j.DefaultPluginManager - PF4J version 3.10.0 in 'deployment' mode
Jul-18 11:07:23.974 [main] INFO  org.pf4j.AbstractPluginManager - No plugins
Jul-18 11:07:24.024 [main] DEBUG nextflow.cli.CmdRun - Applied DSL=2 from script declaration
Jul-18 11:07:24.038 [main] DEBUG nextflow.cli.CmdRun - Launching `scripts/introduction/word_count.nf` [distracted_hodgkin] DSL2 - revision: 2a2e1fc4c0
Jul-18 11:07:24.039 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins default=[]
Jul-18 11:07:24.039 [main] DEBUG nextflow.plugin.PluginsFacade - Plugins resolved requirement=[]
Jul-18 11:07:24.080 [main] DEBUG nextflow.Session - Session UUID: 72f3cfe6-29c4-409d-ad0a-ca41ade4e233
Jul-18 11:07:24.080 [main] DEBUG nextflow.Session - Run name: distracted_hodgkin
Jul-18 11:07:24.081 [main] DEBUG nextflow.Session - Executor pool size: 112
Jul-18 11:07:24.104 [main] DEBUG nextflow.file.FilePorter - File porter settings maxRetries=3; maxTransfers=50; pollTimeout=null
Jul-18 11:07:24.110 [main] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'FileTransfer' minSize=10; maxSize=336; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Jul-18 11:07:24.125 [main] DEBUG nextflow.cli.CmdRun - 
  Version: 24.04.3 build 5916
  Created: 09-07-2024 19:35 UTC (20:35 BST)
  System: Linux 6.5.0-1020-oem
  Runtime: Groovy 4.0.21 on OpenJDK 64-Bit Server VM 11.0.1+13-LTS
  Encoding: UTF-8 (UTF-8)
  Process: 2313682@eb-lnx-0001 [127.0.1.1]
  CPUs: 112 - Mem: 502.9 GB (7.1 GB) - Swap: 2 GB (1.8 GB)
Jul-18 11:07:24.172 [main] DEBUG nextflow.Session - Work-dir: /home/LBEP/Personal/nf-training/work [smb2]
Jul-18 11:07:24.173 [main] DEBUG nextflow.Session - Script base path does not exist or is not a directory: /home/LBEP/Personal/nf-training/scripts/introduction/bin
Jul-18 11:07:24.184 [main] DEBUG nextflow.executor.ExecutorFactory - Extension executors providers=[]
Jul-18 11:07:24.192 [main] DEBUG nextflow.Session - Observer factory: DefaultObserverFactory
Jul-18 11:07:24.217 [main] DEBUG nextflow.cache.CacheFactory - Using Nextflow cache factory: nextflow.cache.DefaultCacheFactory
Jul-18 11:07:24.225 [main] DEBUG nextflow.util.CustomThreadPool - Creating default thread pool > poolSize: 113; maxThreads: 1000
Jul-18 11:07:24.549 [main] DEBUG nextflow.Session - Session start
Jul-18 11:07:24.903 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution
Jul-18 11:07:24.912 [main] DEBUG n.secret.LocalSecretsProvider - Secrets store: /home/.nextflow/secrets/store.json
Jul-18 11:07:24.918 [main] DEBUG nextflow.secret.SecretsLoader - Discovered secrets providers: [nextflow.secret.LocalSecretsProvider@483f286e] - activable => nextflow.secret.LocalSecretsProvider@483f286e
Jul-18 11:07:24.984 [main] DEBUG nextflow.executor.ExecutorFactory - << taskConfig executor: null
Jul-18 11:07:24.984 [main] DEBUG nextflow.executor.ExecutorFactory - >> processorType: 'local'
Jul-18 11:07:24.988 [main] DEBUG nextflow.executor.Executor - [warm up] executor > local
Jul-18 11:07:24.992 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=112; memory=502.9 GB; capacity=112; pollInterval=100ms; dumpInterval=5m
Jul-18 11:07:24.994 [main] DEBUG n.processor.TaskPollingMonitor - >>> barrier register (monitor: local)
Jul-18 11:07:25.077 [main] DEBUG nextflow.Session - Workflow process names [dsl2]: NUM_LINES
Jul-18 11:07:25.077 [main] DEBUG nextflow.Session - Igniting dataflow network (2)
Jul-18 11:07:25.083 [main] DEBUG nextflow.processor.TaskProcessor - Starting process > NUM_LINES
Jul-18 11:07:25.084 [main] DEBUG nextflow.script.ScriptRunner - Parsed script files:
  Script_b3e3e1ab9d817658: /home/LBEP/Personal/nf-training/scripts/introduction/word_count.nf
Jul-18 11:07:25.085 [main] DEBUG nextflow.script.ScriptRunner - > Awaiting termination 
Jul-18 11:07:25.085 [main] DEBUG nextflow.Session - Session await
Jul-18 11:07:25.295 [Task submitter] DEBUG n.executor.local.LocalTaskHandler - Launch cmd line: /bin/bash -ue .command.run
Jul-18 11:07:25.299 [Task submitter] INFO  nextflow.Session - [c8/287802] Submitted process > NUM_LINES (1)
Jul-18 11:07:25.370 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed > TaskHandler[id: 1; name: NUM_LINES (1); status: COMPLETED; exit: 1; error: -; workDir: /home/LBEP/Personal/nf-training/work/c8/28780274129b5b8a351a157551e179]
Jul-18 11:07:25.371 [Task monitor] DEBUG nextflow.util.ThreadPoolBuilder - Creating thread pool 'TaskFinalizer' minSize=10; maxSize=336; workQueue=LinkedBlockingQueue[10000]; allowCoreThreadTimeout=false
Jul-18 11:07:25.379 [TaskFinalizer-1] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=NUM_LINES (1); work-dir=/home/LBEP/Personal/nf-training/work/c8/28780274129b5b8a351a157551e179
  error [nextflow.exception.ProcessFailedException]: Process `NUM_LINES (1)` terminated with an error exit status (1)
Jul-18 11:07:25.391 [TaskFinalizer-1] DEBUG nextflow.processor.TaskRun - Unable to dump output of process 'null' -- Cause: java.nio.file.NoSuchFileException: /home/LBEP/Personal/nf-training/work/c8/28780274129b5b8a351a157551e179/.command.out
Jul-18 11:07:25.395 [TaskFinalizer-1] ERROR nextflow.processor.TaskProcessor - Error executing process > 'NUM_LINES (1)'

Caused by:
  Process `NUM_LINES (1)` terminated with an error exit status (1)

Command executed:

  printf 'ref1_1.fq.gz '
  gunzip -c ref1_1.fq.gz | wc -l

Command exit status:
  1

Command output:
  (empty)

Command error:
  ln: failed to create symbolic link 'ref1_1.fq.gz': Operation not supported

Work dir:
  /home/LBEP/Personal/nf-training/work/c8/28780274129b5b8a351a157551e179

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
Jul-18 11:07:25.403 [main] DEBUG nextflow.Session - Session await > all processes finished
Jul-18 11:07:25.405 [TaskFinalizer-1] DEBUG nextflow.Session - Session aborted -- Cause: Process `NUM_LINES (1)` terminated with an error exit status (1)
Jul-18 11:07:25.416 [main] DEBUG nextflow.Session - Session await > all barriers passed
Jul-18 11:07:25.417 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop
Jul-18 11:07:25.423 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=1; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=0ms; failedDuration=21ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=1; peakCpus=1; peakMemory=0; ]
Jul-18 11:07:25.626 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done
Jul-18 11:07:25.661 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

Thank you

ggrimes commented 1 month ago

Can please temporarily try and older version, and see if it fixes it.

NXF_VER=23.10.0 nextflow run scripts/introduction/word_count.nf
unugwa commented 1 month ago

Still returns the same error