Rosella recover flight status error

[Rridley7]

Hi, I am running into an error when running rosella recover with several metagenomes. The error is not consistent between samples, e.g. I cannot predict when the error will occur, however it does happen consistently on the samples with which it occurs. The error statement is:

Error when running flight process. Exitstatus was : ExitStatus(unix_wait_status(256)) thread 'main' panicked at 'Failed to grab stderr from failed flight process', /home/conda/.cargo/registry/src/github.com-1ecc6299db9ec823/bird_tool_utils-0.3.0/src/command.rs:17:14

The original command was: rosella recover -i S04_1a9817_spa_t_mtb_cov.txt -r S04_1a9817_spa_t_contigs.fa

I can provide the original files if needed, the coverage file was generated by use of coverm contig in metabat mode.

[rhysnewell]

Hmm, yeah that error is not very informative. Would you please provide the reference file and coverage file? I'll see if I can get to the bottom of it

Rhys

[Rridley7]

Files are attached, thanks! S04_1a9817_spa_t_mtb_cov.txt S04_1a9817_spa_t_contigs.fa.zip

[rhysnewell]

Hi!

So i've looked through your files and tried running Rosella on them. You are right that rosella does error out, but I believe it is not due to a problem on rosella's end.

The assembly you are trying to bin is not very good, here are the stats from bbmap for it:

A   C   G   T   N   IUPAC   Other   GC  GC_stdev
0.2519  0.2479  0.2451  0.2551  0.0000  0.0000  0.0000  0.4930  0.0964

Main genome scaffold total:             1828
Main genome contig total:               1828
Main genome scaffold sequence total:    2.745 MB
Main genome contig sequence total:      2.745 MB    0.000% gap
Main genome scaffold N/L50:             663/1.435 KB
Main genome contig N/L50:               663/1.435 KB
Main genome scaffold N/L90:             1562/1.067 KB
Main genome contig N/L90:               1562/1.067 KB
Max scaffold length:                    15.637 KB
Max contig length:                      15.637 KB
Number of scaffolds > 50 KB:            0
% main genome in scaffolds > 50 KB:     0.00%

Minimum     Number          Number          Total           Total           Scaffold
Scaffold    of              of              Scaffold        Contig          Contig
Length      Scaffolds       Contigs         Length          Length          Coverage
--------    --------------  --------------  --------------  --------------  --------
    All              1,828           1,828       2,745,043       2,745,043   100.00%
    500              1,828           1,828       2,745,043       2,745,043   100.00%
   1 KB              1,828           1,828       2,745,043       2,745,043   100.00%
 2.5 KB                 99              99         375,466         375,466   100.00%
   5 KB                 15              15         115,390         115,390   100.00%
  10 KB                  3               3          38,314          38,314   100.00%

As you can see, most of the contigs fall below the default minimum contig size that rosella uses (--min-contig-size 1500). The size of the assembly of contigs > 1Kbp is less than 500Kbp. That's not really a whole lot of information for rosella, or any binning algorithm, to work with. I doubt you will easily get anything informative out of this assembly without some level of manual inspection.

I think I will go ahead and close this issue now. Hopefully you have found my response helpful, and you can find something useful in your assembly.

Cheers, Rhys

[janfelix]

Hello Rhys, I had the same issue and most likely due to the same problem with short contigs. My contigs are assembled from metatranscriptome data, so that's what they are. I had the impression that GroopM was able to process contigs as short as 500bp and then moved on to rosella. Do you see any chance rosella could work with contigs shorter than 1500 bp? Even just to try it out or by only using read coverage...

Thanks again for building rosella and the great support!

[rhysnewell]

You can certainly try it out, you just have to set --min-contig-size to the desired value and see how you go. If it returns and error again, then let me know. Thanks for trying it out :)

You'll probably also want to alter --min-bin-size as well and drop it down to a much lower value if you expect your metaT bins to small

[Rridley7]

This was certainly helpful, thanks!

[janfelix]

Hi, I have tried a few things, contig size and bin size were lowered. Unfortunately, after successfully completing the "Contigs kmers analyzed" part it crashes:

[00:07:56] ⠋ Calculating UMAP embeddings and clustering... 3/6
[2022-11-24T22:17:10Z ERROR bird_tool_utils::command] Error when running flight process. Exitstatus was : ExitStatus(unix_wait_status(256)) thread 'main' panicked at 'Failed to grab stderr from failed flight process', /home/conda/.cargo/registry/src/github.com-1ecc6299db9ec823/bird_tool_utils-0.3.0/src/command.rs:17:14

Not sure what that could mean...

[rhysnewell]

Would you please be able to post the output of conda list for your rosella conda environment?

[janfelix]

Hi, thanks for looking into this!

packages in environment at /home/jan/.conda/envs/rosella:

#

Name Version Build Channel

_libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_kmp_llvm conda-forge asttokens 2.1.0 pyhd8ed1ab_0 conda-forge attrs 22.1.0 pyh71513ae_1 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 pyhd8ed1ab_3 conda-forge backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge biopython 1.80 pypi_0 pypi brotli 1.0.9 h166bdaf_8 conda-forge brotli-bin 1.0.9 h166bdaf_8 conda-forge brotlipy 0.7.0 py39hb9d737c_1005 conda-forge bwa 0.7.17 h7132678_9 bioconda bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.18.1 h7f98852_0 conda-forge ca-certificates 2022.9.24 ha878542_0 conda-forge cachecontrol 0.12.12 pyhd8ed1ab_1 conda-forge cached-property 1.5.2 hd8ed1ab_1 conda-forge cached_property 1.5.2 pyha770c72_1 conda-forge certifi 2022.9.24 pyhd8ed1ab_0 conda-forge cffi 1.15.1 py39h74dc2b5_0
charset-normalizer 2.1.1 pyhd8ed1ab_0 conda-forge colorama 0.4.6 pyhd8ed1ab_0 conda-forge contourpy 1.0.6 py39hf939315_0 conda-forge cryptography 38.0.3 py39hd97740a_0 conda-forge cycler 0.11.0 pyhd8ed1ab_0 conda-forge cython 0.29.32 py39h5a03fae_1 conda-forge dbus 1.13.6 he372182_0 conda-forge decorator 5.1.1 pyhd8ed1ab_0 conda-forge exceptiongroup 1.0.4 pyhd8ed1ab_0 conda-forge executing 1.2.0 pyhd8ed1ab_0 conda-forge expat 2.5.0 h27087fc_0 conda-forge filelock 3.8.0 pyhd8ed1ab_0 conda-forge flight-genome 1.5.0 pypi_0 pypi fontconfig 2.14.1 hc2a2eb6_0 conda-forge fonttools 4.38.0 py39hb9d737c_1 conda-forge freetype 2.12.1 hca18f0e_0 conda-forge glib 2.69.1 h4ff587b_1
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 h28cd5cc_2
h5py 3.7.0 nompi_py39h817c9c5_102 conda-forge hdbscan 0.8.29 pypi_0 pypi hdf5 1.12.2 nompi_h2386368_100 conda-forge hdmedians 0.14.2 py39h2ae25f5_3 conda-forge htslib 1.16 h6bc39ce_0 bioconda icu 58.2 hf484d3e_1000 conda-forge idna 3.4 pyhd8ed1ab_0 conda-forge imageio 2.22.4 pypi_0 pypi iniconfig 1.1.1 pyh9f0ad1d_0 conda-forge ipython 8.4.0 py39hf3d152e_0 conda-forge jedi 0.18.2 pyhd8ed1ab_0 conda-forge joblib 1.1.1 pypi_0 pypi jpeg 9e h166bdaf_2 conda-forge k8 0.2.5 hd03093a_2 bioconda keyutils 1.6.1 h166bdaf_0 conda-forge kiwisolver 1.4.4 py39hf939315_1 conda-forge krb5 1.19.3 h3790be6_0 conda-forge lcms2 2.14 h6ed2654_0 conda-forge ld_impl_linux-64 2.38 h1181459_1
lerc 4.0.0 h27087fc_0 conda-forge libblas 3.9.0 16_linux64_openblas conda-forge libbrotlicommon 1.0.9 h166bdaf_8 conda-forge libbrotlidec 1.0.9 h166bdaf_8 conda-forge libbrotlienc 1.0.9 h166bdaf_8 conda-forge libcblas 3.9.0 16_linux64_openblas conda-forge libcurl 7.86.0 h7bff187_1 conda-forge libdeflate 1.13 h166bdaf_0 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libffi 3.3 he6710b0_2
libgcc-ng 12.2.0 h65d4601_19 conda-forge libgfortran-ng 12.2.0 h69a702a_19 conda-forge libgfortran5 12.2.0 h337968e_19 conda-forge liblapack 3.9.0 16_linux64_openblas conda-forge libllvm11 11.1.0 he0ac6c6_5 conda-forge libnghttp2 1.47.0 hdcd2b5c_1 conda-forge libopenblas 0.3.21 pthreads_h78a6416_3 conda-forge libpng 1.6.39 h753d276_0 conda-forge libssh2 1.10.0 haa6b8db_3 conda-forge libstdcxx-ng 12.2.0 h46fd767_19 conda-forge libtiff 4.4.0 h0e0dad5_3 conda-forge libuuid 2.32.1 h7f98852_1000 conda-forge libwebp-base 1.2.4 h166bdaf_0 conda-forge libxcb 1.13 h7f98852_1004 conda-forge libxml2 2.9.14 h74e7548_0
libzlib 1.2.13 h166bdaf_4 conda-forge llvm-openmp 15.0.5 he0ac6c6_0 conda-forge llvmlite 0.39.1 py39h7d9a04d_1 conda-forge lockfile 0.12.2 py_1 conda-forge matplotlib 3.6.2 py39hf3d152e_0 conda-forge matplotlib-base 3.6.2 py39hf9fd14e_0 conda-forge matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge minimap2 2.24 h7132678_1 bioconda msgpack-python 1.0.4 py39hf939315_1 conda-forge munkres 1.0.7 py_1 bioconda natsort 8.2.0 pyhd8ed1ab_0 conda-forge ncurses 6.3 h5eee18b_3
numba 0.56.4 py39h61ddf18_0 conda-forge numpy 1.21.0 pypi_0 pypi openjpeg 2.5.0 h7d73246_1 conda-forge openssl 1.1.1s h166bdaf_0 conda-forge packaging 21.3 pyhd8ed1ab_0 conda-forge pandas 1.5.2 py39h4661b88_0 conda-forge parallel 20170422 pl5.22.0_0 bioconda parso 0.8.3 pyhd8ed1ab_0 conda-forge patsy 0.5.3 pyhd8ed1ab_0 conda-forge pcre 8.45 h9c3ff4c_0 conda-forge pebble 5.0.3 pypi_0 pypi perl 5.22.0.1 0 conda-forge pexpect 4.8.0 pyh1a96a4e_2 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 9.2.0 py39hf3a2cdf_3 conda-forge pip 22.2.2 py39h06a4308_0
pluggy 1.0.0 pyhd8ed1ab_5 conda-forge prompt-toolkit 3.0.33 pyha770c72_0 conda-forge pthread-stubs 0.4 h36c2ea0_1001 conda-forge ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pygments 2.13.0 pyhd8ed1ab_0 conda-forge pynndescent 0.5.8 pyh1a96a4e_0 conda-forge pyopenssl 22.1.0 pyhd8ed1ab_0 conda-forge pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge pyqt 5.9.2 py39h2531618_6 anaconda pysocks 1.7.1 py39hf3d152e_5 conda-forge pytest 7.2.0 pyhd8ed1ab_2 conda-forge python 3.9.15 haa1d7c7_0
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python_abi 3.9 2_cp39 conda-forge pytz 2022.6 pyhd8ed1ab_0 conda-forge qt 5.9.7 h5867ecd_1
readline 8.2 h5eee18b_0
requests 2.28.1 pyhd8ed1ab_1 conda-forge rosella 0.4.2 h6f8cb4c_1 bioconda samtools 1.16.1 h6899075_1 bioconda scikit-bio 0.5.7 py39hce5d2b2_0 conda-forge scikit-learn 1.0.2 pypi_0 pypi scipy 1.8.1 pypi_0 pypi seaborn 0.12.1 hd8ed1ab_0 conda-forge seaborn-base 0.12.1 pyhd8ed1ab_0 conda-forge setuptools 65.5.0 py39h06a4308_0
sip 4.19.13 py39h295c915_0 anaconda six 1.16.0 pyh6c4a22f_0 conda-forge sqlite 3.39.3 h5082296_0
stack_data 0.6.1 pyhd8ed1ab_0 conda-forge starcode 1.4 hec16e2b_2 bioconda statsmodels 0.13.5 py39h2ae25f5_2 conda-forge tbb 2021.7.1 pypi_0 pypi threadpoolctl 3.1.0 pyh8a188c0_0 conda-forge tk 8.6.12 h1ccaba5_0
tomli 2.0.1 pyhd8ed1ab_0 conda-forge tornado 6.2 py39hb9d737c_1 conda-forge tqdm 4.64.1 pyhd8ed1ab_0 conda-forge traitlets 5.5.0 pyhd8ed1ab_0 conda-forge typing_extensions 4.4.0 pyha770c72_0 conda-forge tzdata 2022f h04d1e81_0
umap-learn 0.5.3 py39hf3d152e_0 conda-forge unicodedata2 15.0.0 py39hb9d737c_0 conda-forge urllib3 1.26.11 pyhd8ed1ab_0 conda-forge wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge wheel 0.37.1 pyhd3eb1b0_0
xorg-libxau 1.0.9 h7f98852_0 conda-forge xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge xz 5.2.6 h5eee18b_0
zlib 1.2.13 h166bdaf_4 conda-forge zstd 1.5.2 h6239696_4 conda-forge