dummy_data file issue - Singularity

[njm-data]

Hi! I'm trying to run the example data (testrun) in Finder on an HPC using Singularity (v 3.7.1). In the main testrun.error file the error that starts the error series was cat: /scratch1/njmello/Finder-finder_v1.1.0/example/testrun/alignments/dummy_data1_round2_SJ.out.tab: No such file or directory

I then went to the progress.log file and saw that the mapping rate for the dummy variables were 0. I went to the alignments folder for the dummy_data1_round1.error and saw

EXITING because of fatal input ERROR: could not open readFilesIn=/home/ubuntu/Finder/example/raw_data//dummy_data1.fastq

Jan 11 13:05:49 ...... FATAL ERROR, exiting

I went to the raw data file and saw this:

[njmello@login001 raw_data]$ pwd
/scratch1/njmello/Finder-finder_v1.1.0/example/raw_data
[njmello@login001 raw_data]$ ls
dummy_data1.fastq  dummy_data2.fastq.gz

I think the double '//' that I saw in the dummy_data1_round1.error is potentially the key to the problem, as I know that the dummy1 fastq file is present. I'll upload the testrun.error file and the progress.log file too if that helps. If there's anything I should try, please let me know. Thank you!! testrun.error.log progress.log

[sagnikbanerjee15]

Hello @njm-data,

Thank you so much for your interest in finder and sorry for the late response. Could you paste the command you are executing? Also, it seems like there were some issues while generating the sif file. The same thing happend with me as well and I had to consult the system administrator to sort it out.

Please let me know if you run into any further issues.

Thank you.

[njm-data]

The command was: run_finder -no_cleanup -mf $PWD/Arabidopsis_thaliana_metadata.csv -n 16 -out_dir $PWD/testrun -g $PWD/Arabidopsis_thaliana.TAIR10.dna_sm.toplevel.fa -p $PWD/uniprot_ARATH.fasta -gm $PWD/../../gmes_linux_64 -gml $PWD/../../gm_key_64 -om PLANTS -fm singularity -preserve 1> $PWD/testrun.output 2> $PWD/testrun.error

On Wed, Jan 12, 2022 at 9:17 PM Sagnik Banerjee @.***> wrote:

Hello @njm-data https://github.com/njm-data,

Thank you so much for your interest in finder and sorry for the late response. Could you paste the command you are executing? Also, it seems like there were some issues while generating the sif file. The same thing happend with me as well and I had to consult the system administrator to sort it out.

Please let me know if you run into any further issues.

Thank you.

— Reply to this email directly, view it on GitHub https://github.com/sagnikbanerjee15/Finder/issues/43#issuecomment-1011682695, or unsubscribe https://github.com/notifications/unsubscribe-auth/APP5XXMPUSGRALGVZPJMEJLUVYY4FANCNFSM5LX2AR6A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

[sagnikbanerjee15]

Hello @njm-data,

Thank you for sending me the command. Could you please tell me what your $PWD is? The last column of the Arabidopsis_thaliana_metadata.csv file stores the location where the data is available. This must match the actual location of the data. Depending on where you download Finder, you may need to change this.

Thank you.

[njm-data]

Hi, my $PWD is /scratch1/njmello/bin/Finder-finder_v1.1.0/example

In regards to contacting the sysadmin about the sif file, is there anything in particular I should ask about?

Thank you so much for your help!

On Thu, Jan 13, 2022 at 11:13 AM Sagnik Banerjee @.***> wrote:

Hello @njm-data https://github.com/njm-data,

Thank you for sending me the command. Could you please tell me what your $PWD is? The last column of the Arabidopsis_thaliana_metadata.csv file stores the location where the data is available. This must match the actual location of the data. Depending on where you download Finder, you may need to change this.

Thank you.

— Reply to this email directly, view it on GitHub https://github.com/sagnikbanerjee15/Finder/issues/43#issuecomment-1012285736, or unsubscribe https://github.com/notifications/unsubscribe-auth/APP5XXKIFCMWMFU4XOPAQC3UV323TANCNFSM5LX2AR6A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

[sagnikbanerjee15]

Hello @njm-data,

Thank you for proving me with that information. Since the $PWD is /scratch1/njmello/bin/Finder-finder_v1.1.0/example, you need to change the final column of the last two rows of metadata.csv file. Just put in "/scratch1/njmello/bin/Finder-finder_v1.1.0/example/raw_data". Please run finder once you make these alterations and see if it works.

While looking at the error file, I noticed two lines "sort: cannot create temporary file in '/local_scratch/pbs.4359122.pbs02': No such file or directory" and "sort: cannot create temporary file in '/local_scratch/pbs.4359122.pbs02': No such file or directory". These might be due to restricted permissions. Singularity tried to write files to some default location to which you may not have write permissions. Best would be to send your system administrator the error file.

Please let me know if you have any questions.

Thank you.

[njm-data]

Changing the meta-data worked! Thank you so much. I'm working with the system administrator about the permissions (which I think ended up causing issues with braker).

Thank you again for your help!!

[sagnikbanerjee15]

Perfect! Feel free to write back to me if you encounter any issues.

Thank you.