iraiosub / riboseq-flow

A Nextflow DSL2 pipeline to perform ribo-seq data analysis and comprehensive quality control.
MIT License
8 stars 1 forks source link

Can't run the test outside the home directory #87

Closed huipan1973 closed 2 months ago

huipan1973 commented 2 months ago

I am running the test of Riboseq-flow on the clusters. I can successfully run the test in my home directory, where the riboseq-flow repo is stored. However, when I change the working directory and run the test, it gives an error saying that a file (multiqc_config.yaml) in the riboseq-flow repo can’t be found. But that file is in the correct location specified by the command shown in the error message. I don’t know what’s going on. It looks like the pipeline can’t work on two directories at the same time.

Also, when running the real data, it gives an error saying the "file argument" can't be null. Possibly, this is due to the same reason.

iraiosub commented 2 months ago

You need to set the environment variable NXF_HOME. NXF_HOME specifies the directory where Nextflow stores its configuration files, logs, and temporary files. By default, Nextflow uses the .nextflow directory in the user’s home directory. By settingNXF_HOME to a custom path, you can control where Nextflow stores these files, which is useful for managing workflows in shared or cluster environments. So when you are outside your home directory you should explicitly set the NXF_HOME variable to a location of your choice that you can re-use for your sub-sequent runs. You can use sth like export NXF_HOME=full_path_of_your_choice/.nextflow. if you don't set this variable, Nextflow will not see the files stored in the repo. You should also always pull the pipeline before running it.

Also make sure your samplesheet contains full paths to the data, otherwise Nextflow would not find the files. This may be the reson for your second error. I recommend you to check the Nextflow documentation or consult your system administrator on how to set it up on your system if this is your first time working with Nextflow pipelines.

huipan1973 commented 2 months ago

Hi,

Thank you for the message. Yes, setting NXF_HOME allow me to run the test outside my home directory.

But I still can’t run the real data even I use full path of all the inputs. It says “ERROR ~ Argument of file function cannot be null” I check the log file, looks like it even can’t download the reference genome.

Here is what is showing in the .nextflow.log

Jun-21 10:15:18.358 [main] DEBUG nextflow.Session - Session await > all processes finished Jun-21 10:15:18.358 [main] DEBUG nextflow.Session - Session await > all barriers passed Jun-21 10:15:18.359 [Task monitor] DEBUG n.processor.TaskPollingMonitor - <<< barrier arrives (monitor: local) - terminating tasks monitor poll loop Jun-21 10:15:18.360 [main] INFO nextflow.Nextflow - -[0;34m[riboseq][1;91m Pipeline completed with errors[0m

Jun-21 10:15:18.361 [Actor Thread 89] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=RIBOSEQ:PREPARE_RIBOSEQ_REFERENCE:GUNZIP_GTF; work-dir=null error [java.lang.InterruptedException]: java.lang.InterruptedException Jun-21 10:15:18.361 [Actor Thread 90] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=RIBOSEQ:PREPARE_RIBOSEQ_REFERENCE:GUNZIP_CONTAMINANTS_FASTA; work-dir=null error [java.lang.InterruptedException]: java.lang.InterruptedException Jun-21 10:15:18.361 [Actor Thread 88] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for task: name=RIBOSEQ:PREPARE_RIBOSEQ_REFERENCE:GUNZIP_FASTA; work-dir=null error [java.lang.InterruptedException]: java.lang.InterruptedException Jun-21 10:15:18.373 [main] DEBUG n.trace.WorkflowStatsObserver - Workflow completed > WorkflowStats[succeededCount=0; failedCount=0; ignoredCount=0; cachedCount=0; pendingCount=0; submittedCount=0; runningCount=0; retriesCount=0; abortedCount=0; succeedDuration=0ms; failedDuration=0ms; cachedDuration=0ms;loadCpus=0; loadMemory=0; peakRunning=0; peakCpus=0; peakMemory=0; ] Jun-21 10:15:18.373 [main] DEBUG nextflow.trace.TraceFileObserver - Workflow completed -- saving trace file Jun-21 10:15:18.376 [main] DEBUG nextflow.trace.ReportObserver - Workflow completed -- rendering execution report Jun-21 10:15:18.403 [Actor Thread 236] DEBUG nextflow.container.SingularityCache - Singularity found local store for image=https://depot.galaxyproject.org/singularity/ubuntu:20.04; path=/n/jobspace/bbcore/schaffer_rpf_abby_min6/riboseq_flow/.singularity/cache/depot.galaxyproject.org-singularity-ubuntu-20.04.img Jun-21 10:15:18.418 [Actor Thread 128] DEBUG nextflow.util.HashBuilder - Hash asset file sha-256: /n/jobspace/bbcore/schaffer_rpf_abby_min6/riboseq_flow/.nextflow/assets/iraiosub/riboseq-flow/assets/contaminants/GRCm38.p6.snRNA_snoRNA_rRNA_with_full_rRNA_OSCAR_fixed.fa.gz Jun-21 10:15:18.879 [FileTransfer-2] DEBUG nextflow.file.FilePorter - Copying foreign file https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_mouse/release_M33/gencode.vM33.primary_assembly.annotation.gtf.gz to work dir: /n/jobspace/bbcore/schaffer_rpf_abby_min6/riboseq_flow/work/stage-0c3ed245-b9b9-46f1-bf60-6549d2e2041d/c4/42619573eb42a1fb585e49e04c8744/gencode.vM33.primary_assembly.annotation.gtf.gz Jun-21 10:15:18.879 [FileTransfer-1] DEBUG nextflow.file.FilePorter - Copying foreign file https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_mouse/release_M33/GRCm39.primary_assembly.genome.fa.gz to work dir: /n/jobspace/bbcore/schaffer_rpf_abby_min6/riboseq_flow/work/stage-0c3ed245-b9b9-46f1-bf60-6549d2e2041d/47/86d415e4c09496e7d2a6c924594365/GRCm39.primary_assembly.genome.fa.gz Jun-21 10:15:19.109 [main] DEBUG nextflow.trace.TimelineObserver - Workflow completed -- rendering execution timeline Jun-21 10:15:19.332 [main] DEBUG nextflow.cache.CacheDB - Closing CacheDB done Jun-21 10:15:19.353 [main] DEBUG nextflow.script.ScriptRunner - > Execution complete -- Goodbye

Best regards, Hui

From: iraiosub @.> Sent: Friday, June 21, 2024 9:32 AM To: iraiosub/riboseq-flow @.> Cc: Hui Pan @.>; Author @.> Subject: Re: [iraiosub/riboseq-flow] Can't run the test outside the home directory (Issue #87)

You need to set the environment variable NXF_HOME. NXF_HOME specifies the directory where Nextflow stores its configuration files, logs, and temporary files. By default, Nextflow uses the .nextflow directory in the user’s home directory. By setting NXF_HOME to a custom path, you can control where Nextflow stores these files, which is useful for managing workflows in shared or cluster environments. So when you are outside your home directory you should explicitly set the NXF_HOME variable to a location of your choice that you can re-use for your sub-sequent runs. You can use sth like export NXF_HOME=full_path_of_your_choice/.nextflow. if you don't set this variable, Nextflow will not see the files stored in the repo. You should also always pull the pipeline before running it.

Also make sure your samplesheet contains full paths to the data, otherwise Nextflow would not find the files. This may be the reson for your second error. I recommend you to check the Nextflow documentation or consult your system administrator on how to set it up on your system if this is your first time working with Nextflow pipelines.

— Reply to this email directly, view it on GitHubhttps://github.com/iraiosub/riboseq-flow/issues/87#issuecomment-2182762209, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABBEJDVYKSEK5SHO5LRPFXTZIQTMJAVCNFSM6AAAAABJV4BGM2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBSG43DEMRQHE. You are receiving this because you authored the thread.Message ID: @.**@.>>