Input file structure and file type

[joyleng]

Ask away!

Hi

Starting a new question as it is a different issue to the one I previously put this at the bottom of.

I have just updated EPI2ME to 5.1.10 but my workflow seems to be getting stuck at the early stages of analysis. I am using the same file structure that I used previously: I am pointing EPI2ME to a folder that contains a folder per sample (called "barcode01", "barcode02" etc.) and within these files are the compressed fastq files the that came off the MinION. I can see them as compressed fastq files on my windows file exploring so I am presuming these would be seen as fastq.gz?? At the moment I have just included one folder called "barcode01" in the folder for test purposes but once it is working the folder will contain 94 barcode files. I read through the page on "input directory structure" but I feel like I am missing something.

Here is my error log: N E X T F L O W ~ version 23.04.2 Launching /mnt/c/Users/joyle/epi2melabs/workflows/epi2me-labs/wf-16s/main.nf [friendly_agnesi] DSL2 - revision: 7338c9bcbe WARN: NEXTFLOW RECURSION IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES ERROR ~ Validation of pipeline parameters failed! -- Check '/mnt/c/Users/joyle/epi2melabs/instances/wf-16s_01HQJE2PFKAYQ1CTBJ2PT70NKE/nextflow.log' file for details ERROR ~ * Bad parameter configuration. You must select only one option of: --fastq --bam -- Check '/mnt/c/Users/joyle/epi2melabs/instances/wf-16s_01HQJE2PFKAYQ1CTBJ2PT70NKE/nextflow.log' file for details WARN: Found unexpected parameters:

• --client_fields: C:\Users\joyle\epi2melabs\instances\wf-16s_01HQJE2PFKAYQ1CTBJ2PT70NKE\client_fields.json
• Ignore this warning: params.schema_ignore_params = "client_fields"

I used the top argument in "input options" to point EPI2ME to the fastq files as I noticed you can now point it to BAM files below this as well. As I do not have BAM files I left this blank. Hopefully I am just missing something simple.

Thanks Joy

[nggvs]

Hi @joyleng ,

I'll take a look on this issue, meanwhile, could you confirm that:

• you are able to run the demo?
• could you run the wf using as input the next folder:
  • Go to Setting in EPI2ME -> working directory -> copy the path and move there in the windows file explorer. Once you are there, go to workflows -> epi2me-labs -> wf-metagenomics -> test_data -> case01

In theory the workflow accept [.fastq, .fastq.gz, .fq, .fq.gz] so your files are ok, in particular if you have used them successfully with other versions.
Regarding BAM input, you shouldn't be worried about this, you did well, just used the fastq and leave bam blank.

[nggvs]

Hi @joyleng , Which is the path to your files? For instance, I got the same error when I try to use files that are in my Ubuntu distro within WSL.

[joyleng]

Hi

Thanks for getting back to me. In answer to your questions:

I just tried to run the demos on the 16S and the metagenomic workflows and they both error with a similar message about not having enough RAM (so the issue I had opened previously). Here's the one from the metagenomics wf: ERROR ~ Consider to use --kraken2_memory_mapping to reduce the use of RAM memory. -- Check '/mnt/c/Users/joyle/epi2melabs/instances/wf-metagenomics_01HQXDX13M1GDAJA2DFSKCTP6S/nextflow.log' file for details ERROR ~ Error executing process > 'kraken_pipeline:run_kraken2 (1)' Caused by: Process requirement exceeds available memory -- req: 4 GB; avail: 3.7 GB Command executed: kraken2 --db ncbi_targeted_loci_kraken2_db seqs.fastq.gz --threads 4 --report "barcode03.kraken2.report.txt" --confidence 0 > "barcode03.kraken2.assignments.tsv" Command exit status:

I tried the workflow using the following folder as the input folder: C:\Users\joyle\epi2melabs\workflows\epi2me-labs\wf-metagenomics\test_data\case01 and it gives the same error as using the demo data.

Thanks Joy

[joyleng]

Sorry just seen you new message. The files are on an external hard-drive as I don't have enough space on my computer to store all my sequencing files. Do you want the full path to where sequencing my files are?

Thanks Joy

[nggvs]

Hi @joyleng , No need to have the full path. Thanks for confirming that your samples are in a hard-drive. Also, regarding the issue with the memory from the previous output, which is the RAM memory of your device? Thank you very much in advance!

[nggvs]

Hi @joyleng , Regarding the use of a hard-drive: at this moment is not possible to use samples stored in an external hard-drive, but this feature has already been considered and would be available in future releases. Apologies for the inconvenience.

[joyleng]

Hi

Ah Ok that makes sense about the files being on an external hard-drive. It was running previously when the files were stored on the external hard-drive and I pointed EPI2ME to that folder.

In terms of RAM my laptop has 8GB of RAM available. I tried a couple of fixes to up the limit of RAM that was imposed but nothing seem to work. I don't know if you have a work around to tell EPI2ME that there is more available RAM on the laptop?

Thanks again Joy

[nggvs]

Hi @joyleng , Regarding to your question, you can change the memory available in WSL in the latest version of the desktop app EPI2ME going to Settings > WSL Config

This would fix the problem: req: 4 GB; avail: 3.7 GB

Please let me know if that fixes the issue and thank you for using the workflow

[joyleng]

Hi

I have just tried the above but the option keeps reverting back to 75%. I changed the drop down and clicked save config but if I go out of the settings tab and go back in I can still see the below:

Thanks Joy

[nggvs]

Hi @joyleng ,

Don't worry, I tried it and the config is actually changed (although you don't see it there). Could you retry the workflow and see if now it runs?

[joyleng]

Hi

Just tested it again as it wasn't working yesterday but seems to be fine now. I have just tested it on a single sample stored locally so will test it on a larger number of samples to double check it is ok.

Thanks for all your help Joy

[nggvs]

Hi @joyleng , Thanks for the update! Would you mind to close the issue once you double check is ok? Thank you very much in advance

[nggvs]

Hi @joyleng , I'm going to close the issue as from your last message I guess the workflow is working. Thank you very much for using it and please feel free to report any other issue that you find!