q2-mOTUs installation - Githubissues

ARW-UBT commented 1 year ago

Hello, I have installed q2-mOTUs on q2-2023.2 (and also tried q2-2022.8) as instructed. The installation proceeded to the end, but the motus pligin is not available in qiime.

I have attached the screen output, could you have a look at it whether you can detect any issue there.

Best,

q2-mOTUS-Installation-q2-2023-2.txt

valentynbez commented 1 year ago

Could you please specify the command you are running to run motus in qiime? And what output do you get? From the log, the installation is complete.

Cheers Valentyn

ARW-UBT commented 1 year ago

The command would be qiime motus which however does not work since there is no plugin installed!

The plugins can be listed by qiime The output showes that there is no motus plugin available. (see below, this is a mix of q2 core plugins and some others from q2 library)

################## Usage: qiime [OPTIONS] COMMAND [ARGS]...

QIIME 2 command-line interface (q2cli)

(remove some part here)

Options: --version Show the version and exit. --help Show this message and exit.

Commands: info Display information about current deployment. tools Tools for working with QIIME 2 files. dev Utilities for developers and advanced users. SCNIC Plugin for SCNIC usage. alignment Plugin for generating and manipulating alignments. clawback CLAss Weight Assembler plugin. composition Plugin for compositional data analysis. cutadapt Plugin for removing adapter sequences, primers, and other unwanted sequence from sequence data. dada2 Plugin for sequence quality control with DADA2. deblur Plugin for sequence quality control with Deblur. deicode Plugin for Robust Aitchison PCA:compositional biplots from sparse count data. demux Plugin for demultiplexing & viewing sequence quality. diversity Plugin for exploring community diversity. diversity-lib Plugin for computing community diversity. emperor Plugin for ordination plotting with Emperor. empress Plugin for visualizing phylogenies with Empress. feature-classifier Plugin for taxonomic classification. feature-table Plugin for working with sample by feature tables. fragment-insertion Plugin for extending phylogenies. gneiss Plugin for building compositional models. krona Plugin for creating Krona plots. longitudinal Plugin for paired sample and time series analyses. metadata Plugin for working with Metadata. phylogeny Plugin for generating and manipulating phylogenies. quality-control Plugin for quality control of feature and sequence data. quality-filter Plugin for PHRED-based filtering and trimming. qurro Plugin for visualizing feature rankings and log-ratios. rescript Pipeline for reference sequence annotation and curation. sample-classifier Plugin for machine learning prediction of sample metadata. shogun Shallow shotgun sequencing taxonomy profiler. taxa Plugin for working with feature taxonomy annotations. vsearch Plugin for clustering and dereplicating with vsearch.

valentynbez commented 1 year ago

Could you please add the steps, by which you install the plugin? I'll try to recreate your error and fix the error :)

ARW-UBT commented 1 year ago

Hi, I followed the instructions given at https://library.qiime2.org/plugins/motus/45/ Qiime2 conda environment was acttivated before executing the following steps.

git clone https://github.com/motu-tool/q2-mOTUs cd q2-mOTUs make install

ARW-UBT commented 1 year ago

Hi @valentynbez I accidentally closed (and reopened) this post. It is still active. I would appreciate if you could find a solution for the installation of the plugin to qiime2. Best regards,

valentynbez commented 1 year ago

Hello @ARW-UBT! We updated the installation and moved it completely to conda. Unfortunately, the previous way didn't account for zsh specifics. Could you please test the new way on your end?

ARW-UBT commented 1 year ago

Hi @valentynbez Thank you for your update. I have tried to install it in qiime2-2023.2 using mamba: conda install mamba -c conda-forge followed by: mamba install -c bioconda -c conda-forge -c valentynbez q2-motus

I got this screen output. It says that r-base.. requires libglib. I looked by conda list, whether they are installed. r-base is there, but libglib was missing. I installed libglib in conda base: conda list showes... libglib 2.74.1 h606061b_1 conda-forge r-base 4.2.2 h6b4767f_2 conda-forge

Then, I activated qiime2-2023.2 again and re-run the mamba (...) q2-motus again, but I get the same error message (see below).

Any idea what could be wrong? Best regards,

mamba outout: (...) Looking for: ['q2-motus']

valentynbez/linux-64 No change valentynbez/noarch No change pkgs/r/linux-64 No change pkgs/r/noarch No change bioconda/noarch 4.2MB @ 3.9MB/s 1.2s pkgs/main/linux-64 No change pkgs/main/noarch No change bioconda/linux-64 4.6MB @ 3.4MB/s 1.4s conda-forge/noarch 11.7MB @ 4.5MB/s 2.7s conda-forge/linux-64 30.5MB @ 4.6MB/s 7.1s

Pinned packages:

python 3.8.*

Could not solve for environment specs Encountered problems while solving:

package r-base-4.2.2-h6b4767f_2 requires libglib >=2.74.1,<3.0a0, but none of the providers can be installed

The environment can't be solved, aborting the operation

ARW-UBT commented 1 year ago

Short additional question: I have read at mamba.readthedocs.io that "Installing mamba into any other environment than base is not supported." I assume it would be ok if mamba is in the base env, not in a specific qiime2 env?

valentynbez commented 1 year ago

Yes, it's okay to have mamba anywhere. However, we encountered dependency issues between QIIME 2 and mOTUs that are impossible to solve within conda. We decided to fall back on Makefile installation. Please, test the updated installation procedure once again :)

ARW-UBT commented 1 year ago

Great, it worked now! Thank you again for the updates. FYI: I Installed it in q2-2022.11, which I have not tested before. The q2-2023.2 env already had a bwa folder in the miniconda/bin, and I will first have to figure out if I can simply rename/move this folder, or whether I have to recreate the 2023.2 env from the beginning. (No problem, it will take just a few minutes to reinstall it again).

qiime motus profile seems to invoke the plugin as normal. However, there were two warnings during installation; I include these few lines here for you (some parts are in German):

bwt_gen.c: In Funktion »BWTIncBuildRelativeRank«: bwt_gen.c:879:17: Warnung: Variable »oldInverseSa0RelativeRank« gesetzt, aber nicht verwendet [-Wunused-but-set-variable] 879 | bgint_t oldInverseSa0RelativeRank = 0; | ^~~~~~~~~ bwt_gen.c: In Funktion »BWTIncMergeBwt«: bwt_gen.c:953:22: Warnung: Variable »bitsInWordMinusBitPerChar« gesetzt, aber nicht verwendet [-Wunused-but-set-variable] 953 | unsigned int bitsInWordMinusBitPerChar; | ^~~~~~~~~

But at the end, it seems that everything has been installed:

| SETUP MOTUS TOOL |

Download the compressed motus database (~3.5Gb) 100%, 3342 MB, 17140 KB/s, 199 seconds passed

Check md5: MD5 verified Extract files from the archive...done Remove zipped file...done Add version file...done

pip install . Processing /home/bt140047/q2-mOTUs Preparing metadata (setup.py) ... done Requirement already satisfied: motu-profiler in /home/bt140047/miniconda3/envs/qiime2-2022.11/lib/python3.8/site-packages (from q2-motus==0+untagged.37.g241e285) (3.0.3) Building wheels for collected packages: q2-motus Building wheel for q2-motus (setup.py) ... done Created wheel for q2-motus: filename=q2_motus-0+untagged.37.g241e285-py3-none-any.whl size=149732 sha256=a0a7f1abab51ef0a0b667a2e3ffead27046008083ca869fc8f12cf5b6fe99fd1 Stored in directory: /home/bt140047/.cache/pip/wheels/f7/7c/d8/14d9592d34100d687b3b7950e6e23bbd632767a7e3907342f5 Successfully built q2-motus Installing collected packages: q2-motus Successfully installed q2-motus-0+untagged.37.g241e285 $

I will test it with own date in the next few days. Thanks again for your fast support!

valentynbez commented 1 year ago

This errors come from bwa we use for sequence alignment, they are not critical though. Thanks for using the tool and cooperation during this issue! :) I will close the issue, but feel free to update the topic with your results!

ARW-UBT commented 1 year ago

Hi @valentynbez I installed q2-motus in a fresh qiime2-2023.2 environment, but I had to 'git clone' the installation directory again, since some installation directories (or symlinks) of a previous setup for q2-2022.11 prevented the installation script from completetion. I have a few questions: Would you advice to 'git clone' q2-motus again when new qiime2-versions will be installed? Is the q2-mOTUs installation directory required after the plugin has been added to the qiime environment (I am not sure where the motus database is installed)? Do you plan to update motus databases on a regular scheme? q2-motus contains optional parameters for --p-threads and --p-jobs: Is there an 'optimal' combination of these two parameters? I do have 40 cpus available, and I used 8 threads and 5 jobs for a run composed of ca. 300 million reads from 9 samples (duration: 7 h 30 min).

In addition to your tutorial, I have looked into my data with some other tools such as robust Aitchison distance (deicode, qurro), aldex2, ancombc, and the results are very helpful in interpreting the data.

Thank you again for this great tool to analyze metagenome sequencing data. Best,

valentynbez commented 1 year ago

Assuming you work in the same conda environment, there is no need to start from git clone, but you will need to install the tool with make install again.
Repo is not required after installation, the database is installed into your environment directory and is hidden there.
Yes, we do plan to update the database regularly, with release 3.1 coming soon & 4.0 coming next year with updated software.
Basically q2-motus runs multiple instances of motu profile using multiprocessing library. motu profile scales effectively until 4-8 threads. In your case, I would run 8 threads x 4 jobs because there are a lot of reads to be aligned to a database in a single sample.
If you can post your results using other QIIME 2 plugins that would be really helpful to show the versatility of the tool. We can open a separate issue for that :)

ARW-UBT commented 1 year ago

The data are not yet published, but I will let you know when this will be done... :-)

motu-tool / q2-mOTUs

q2-mOTUs installation #3

QIIME 2 command-line interface (q2cli)

| SETUP MOTUS TOOL |