search

borenstein-lab / MetaLAFFA

A bioinformatics pipeline for annotating functional capacities in shotgun metagenomic data with native compute cluster integration
GNU General Public License v3.0
7 stars 3 forks source link

Issue with Ortholog abundance correction #8

Open santoshe1 opened 3 years ago

santoshe1 commented 3 years ago

Hello,

I am getting an error shown below with the test dataset provided by Metalaffa. I tried increasing the latency period to 180 seconds and still I face the same error:-

Running MUSiCC... Loading data using pandas module... Traceback (most recent call last): File "/miniconda3/envs/metalaffa/bin/run_musicc.py", line 26, in <module> correct_and_normalize(vars(given_args)) File "/miniconda3/envs/metalaffa/lib/python3.6/site-packages/musicc/core.py", line 208, in correct_and_normalize genes = genes[genes_sort_ind] IndexError: arrays used as indices must be of integer (or boolean) type gzip: /tmp//ortholog_abundance_correction/orthologs.tab: No such file or directory mv: cannot stat '/tmp//ortholog_abundance_correction/orthologs.tab.gz': No such file or directory Error in job ortholog_abundance_correction while creating output file output/ortholog_abundance_correction/host_filter_type_default/duplicate_filter_type_default/quality_filter_type_default_max_info_90:0.7_min_len_60/map_reads_to_genes_type_default_method_blastx_top_percentage_1_evalue_cutoff_0.001_sensitivity_/hit_filter_type_default_method_best_hit/hit_filter_combine_type_default/gene_map_type_default_method_fractional/ortholog_map_type_default_method_fractional/ortholog_map_combine_type_default/ortholog_abundance_correction_type_default_method_musicc_musicc_method_['-n',_'-c',_'learn_model']/orthologs.tab.gz. MissingOutputException in line 357 of /scratch_ws/santosh_ws_env/metalaffa/metalaffa_test_example/Snakefile: Missing files after 5 seconds: output/ortholog_abundance_correction/host_filter_type_default/duplicate_filter_type_default/quality_filter_type_default_max_info_90:0.7_min_len_60/map_reads_to_genes_type_default_method_blastx_top_percentage_1_evalue_cutoff_0.001_sensitivity_/hit_filter_type_default_method_best_hit/hit_filter_combine_type_default/gene_map_type_default_method_fractional/ortholog_map_type_default_method_fractional/ortholog_map_combine_type_default/ortholog_abundance_correction_type_default_method_musicc_musicc_method_['-n',_'-c',_'learn_model']/orthologs.tab.gz This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait. Exiting because a job execution failed. Look above for error message

engal commented 3 years ago

It looks like the issue is occurring when running MUSiCC (see first couple of lines in the traceback). When you say test dataset provided by MetaLAFFA, do you mean you downloaded one or more samples from HMP? Also, could you share the file that was being used as input to the abundance correction step? You should be able to find it in the following directory:

output/ortholog_map_combine/host_filter_type_default/duplicate_filter_type_default/quality_filter_type_default_max_info_90:0.7_min_len_60/map_reads_to_genes_type_default_method_blastx_top_percentage_1_evalue_cutoff_0.001_sensitivity_/hit_filter_type_default_method_best_hit/hit_filter_combine_type_default/gene_map_type_default_method_fractional/ortholog_map_type_default_method_fractional/ortholog_map_combine_type_default/
santoshe1 commented 3 years ago

Hello Engal,

Thank you for the response.

I am referring to the test dataset which you have provided in metalaffa page - <wget http://downloads.hmpdacc.org/data/Illumina/stool/SRS017307.tar.bz2 >

I am also attaching the requested input file to abundance correction step file. orthologs.tab.gz

engal commented 3 years ago

Thanks for the additional info. I suspect that is is related to https://github.com/borenstein-lab/MetaLAFFA/issues/7, so once that issue is resolved, I think you should try rerunning the prepare_databases.py script and then rerunning MetaLAFFA on your data. I'll keep you updated on the status of that issue.

engal commented 3 years ago

Sorry for the delay. I've just released an update for MetaLAFFA that should address the underlying issue. Please let me know if the issue persists or if there are any further issues.