sekhwal commented 4 years ago

Hi, I am not sure which value I need to give here to configure pipeline..Please suggest...

A conda environment named MetONTIIME_env is created, where seqtk, porechop, pycoQC, NanoFilt and qiime2-2019.10 are installed. Then, you can open the config_MinION_mobile_lab.R file with a text editor and set the variables PIPELINE_DIR and MINICONDA_DIR to the value suggested by the installation step.

MaestSi commented 4 years ago

Hi, after following the installation steps, you should set PIPELINE_DIR and MINICONDA_DIR to the paths corresponding to MetONTIIME dir and miniconda3 dir. For example, in my installation I set the following values:

#PIPELINE DIR
PIPELINE_DIR <- "/home/simone/MinION/MinION_scripts/MetONTIIME"
#MINICONDA DIR
MINICONDA_DIR <- "/home/simone/miniconda3"

In the config file, these are not the only 2 values you have to set. However, based on the error you reported in the QIIME2 forum, it looks like you have not run the installation step yet. Can you please confirm this? Let me know if you have any issues running the installation steps. Simone

sekhwal commented 4 years ago

Hi, I have done installation steps. Please see my configuration in the file "config_MinION_mobile_lab". However, still it is showing same error, I am suspect if it for guppy? files

######################################################################################################## PIPELINE_DIR <- "/home/kumarm/MinION-data/MetONTIIME"

MINICONDA DIR

MINICONDA_DIR <- "/home/kumarm/miniconda3"

basecaller_dir

BASECALLER_DIR <- "/home/kumarm/MinION-data/ont-guppy-cpu/guppy-0.1.11/"

NCBI-downloaded sequences (QIIME2 artifact)

DB <- "/path/to/PRJNA33175_Bacterial_sequences.qza"

Taxonomy of NCBI-downloaded sequences (QIIME2 artifact)

TAXONOMY <- "/path/to/PRJNA33175_taxonomy.qza"

sample-metadata file describing samples metadata; it is created automatically if it doesn't exist

SAMPLE_METADATA <- "/path/to/sample-metadata.tsv" ########################################################################################################

load BioStrings package

suppressMessages(library(Biostrings))

path to MetONTIIME.sh

MetONTIIME <- paste0(PIPELINE_DIR, "/home/kumarm/MinION-data/MetONTIIME/MetONTIIME.sh")

path to subsample fast5

subsample_fast5 <- paste0(PIPELINE_DIR, "/home/kumarm/MinION-data/MetONTIIME/subsample_fast5.sh") #########################################################################################################

SEQTK

SEQTK <- paste0(MINICONDA_DIR, "/home/kumarm/miniconda3/envs/MetONTIIME_env/bin/seqtk")

PORECHOP

PORECHOP <- paste0(MINICONDA_DIR, "/home/kumarm/miniconda3/envs/MetONTIIME_env/bin/porechop")

PYCOQC

PYCOQC <- paste0(MINICONDA_DIR, "/home/kumarm/miniconda3/envs/MetONTIIME_env/bin/pycoQC")

NANOFILT

NANOFILT <- paste0(MINICONDA_DIR, "/home/kumarm/miniconda3/envs/MetONTIIME_env/bin/NanoFilt")

MaestSi commented 4 years ago

Hi, the only variables you have to modify are in the

PIPELINE_DIR <- "/home/kumarm/MinION-data/MetONTIIME"
#MINICONDA DIR
MINICONDA_DIR <- "/home/kumarm/miniconda3"
#basecaller_dir
BASECALLER_DIR <- "/home/kumarm/MinION-data/ont-guppy-cpu/guppy-0.1.11/"
#NCBI-downloaded sequences (QIIME2 artifact)
DB <- "/path/to/PRJNA33175_Bacterial_sequences.qza"
#Taxonomy of NCBI-downloaded sequences (QIIME2 artifact)
TAXONOMY <- "/path/to/PRJNA33175_taxonomy.qza"
#sample-metadata file describing samples metadata; it is created automatically if it doesn't exist
SAMPLE_METADATA <- "/path/to/sample-metadata.tsv"

box (and in the above part of the script, for setting the desired options). I see you have edited also SEQTK, PORECHOP, PYCOQC and NANOFILT values, but those values should have been left unchanged. So, my advice is to copy the original value for all variables that are not in that box. Moreover, I notice that guppy v0.1.11 is a very old version, I think you need to download a newer one. After you have done this, can you please tell me if the ./install.sh instruction gave any errors?

sekhwal commented 4 years ago

there are no any errors while running ./install.sh

I have improved my configuration in the file.."config_MinION_mobile_lab". However it is showing same error.....

######################################################################################################## PIPELINE_DIR <- "/home/kumarm/MinION-data/MetONTIIME"

MINICONDA DIR

MINICONDA_DIR <- "/home/kumarm/miniconda3/miniconda3"

basecaller_dir

BASECALLER_DIR <- "/home/kumarm/MinION-data/ont-guppy-cpu/guppy-0.1.11/"

NCBI-downloaded sequences (QIIME2 artifact)

DB <- "/path/to/PRJNA33175_Bacterial_sequences.qza"

Taxonomy of NCBI-downloaded sequences (QIIME2 artifact)

TAXONOMY <- "/path/to/PRJNA33175_taxonomy.qza"

sample-metadata file describing samples metadata; it is created automatically if it doesn't exist

SAMPLE_METADATA <- "/path/to/sample-metadata.tsv" ########################################################################################################

load BioStrings package

suppressMessages(library(Biostrings))

path to MetONTIIME.sh

MetONTIIME <- paste0(PIPELINE_DIR, "/MetONTIIME.sh")

path to subsample fast5

subsample_fast5 <- paste0(PIPELINE_DIR, "/subsample_fast5.sh") #########################################################################################################

SEQTK

SEQTK <- paste0(MINICONDA_DIR, "/envs/MetONTIIME_env/bin/seqtk")

PORECHOP

PORECHOP <- paste0(MINICONDA_DIR, "/envs/MetONTIIME_env/bin/porechop")

PYCOQC

PYCOQC <- paste0(MINICONDA_DIR, "/envs/MetONTIIME_env/bin/pycoQC")

NANOFILT

NANOFILT <- paste0(MINICONDA_DIR, "/envs/MetONTIIME_env/bin/NanoFilt")

MaestSi commented 4 years ago

Hi, MINICONDA_DIR <- "/home/kumarm/miniconda3/miniconda3" should probably be changed to MINICONDA_DIR <- "/home/kumarm/miniconda3". Then, as I told you, you should update Guppy to v3.4. Could you please report the error you had when running Import_database.sh?

MaestSi commented 4 years ago

Based on the error you posted on QIIME2 forum, it looks like the install.sh script was not run successfully yet, as there was error Could not find conda environment: entrez_qiime_env.

sekhwal commented 4 years ago

Yes! This is the error one. Is it installation error? It occur while running Import_database.sh. Please see below error message..

(base) kumarm@microbiome:~/MinION-data/MetONTIIME$ ./Import_database.sh sequence.fasta

NotWritableError: The current user does not have write permissions to a required path. path: /anaconda3/envs/.conda_envs_dir_test uid: 1024 gid: 1023

If you feel that permissions on this path are set incorrectly, you can manually change them by executing

$ sudo chown 1024:1023 /anaconda3/envs/.conda_envs_dir_test

In general, it's not advisable to use 'sudo conda'.

Could not find conda environment: entrez_qiime_env You can list all discoverable environments with conda info --envs.

Requirement already satisfied: numpy in /home/kumarm/miniconda3/lib/python3.7/site-packages (1.18.0) Collecting cogent Using cached https://files.pythonhosted.org/packages/03/9e/a7156965da5edc34c3c07661c3b4bc4259caa3d9dcecaa5f7f3ff51eecd7/cogent-1.9.tar.gz ERROR: Command errored out with exit status 1: command: /home/kumarm/miniconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-jw63tuu2/cogent/setup.py'"'"'; file ='"'"'/tmp/pip-install-jw63tuu2/cogent/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.c lose();exec(compile(code, file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-jw63tuu2/cogent/pip-egg-info cwd: /tmp/pip-install-jw63tuu2/cogent/ Complete output (6 lines): Traceback (most recent call last): File "", line 1, in File "/tmp/pip-install-jw63tuu2/cogent/setup.py", line 61 print "Failed to build html due to ImportErrors for sphinx" ^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Failed to build html due to ImportErrors for sphinx")?

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. File "./entrez_qiime/entrez_qiime.py", line 192 print "Need to supply either FASTA file or accession number list as input." # should never get here because either -i or -L is a required argument. ^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Need to supply either FASTA file or accession number list as input." # should never g et here because either -i or -L is a required argument.)? Imported sequence.fasta as DNAFASTAFormat to /home/kumarm/MinION-data/MetONTIIME/sequence_sequence.qza Usage: qiime tools import [OPTIONS]

Import data to create a new QIIME 2 Artifact. See https://docs.qiime2.org/ for usage examples and details on the file types and associated semantic types that can be imported.

Options: --type TEXT The semantic type of the artifact that will be created upon importing. Use --show-importable-types to see what importable semantic types are available in the current deployment. [required] --input-path PATH Path to file or directory that should be imported. [required] --output-path ARTIFACT Path where output artifact should be written. [required] --input-format TEXT The format of the data to be imported. If not provided, data must be in the format expected by the semantic type provided via --type. --show-importable-types Show the semantic types that can be supplied to --type to import data into an artifact. --show-importable-formats Show formats that can be supplied to --input-format to import data into an artifact. --help Show this message and exit.

                There was a problem with the command:

(1/1) Invalid value for "--input-path": Path "/home/kumarm/MinION- data/MetONTIIME/sequence_accession_taxonomy.txt" does not exist.

sekhwal commented 4 years ago

When I run following commands there are no any error generates..and entrez_qiime directory generate after running Import_database.sh sequence.fasta command, then it shows above that error..

git clone https://github.com/MaestSi/MetONTIIME.git cd MetONTIIME chmod 755 * ./install.sh

sekhwal commented 4 years ago

I have updated guppy3-3.0.9 but not sure where I should find v3.4. I think the error is not by guppy version..

MaestSi commented 4 years ago

Hi @sekhwal , this error is not generated by Guppy version for sure, I was just thinking about future configuration steps. It looks like the current error is caused by the impossibility to create a virtual environment named entrez_qiime_env, which is based on python 2. The Syntax error you get is because the workflow is trying to run the code in another environment based on python 3. According to the errors, it looks like you also have anaconda installed, in addition to miniconda3. Is it the case? My guess is that anaconda/bin dir comes in the PATH environment variable before miniconda3/bin dir and they may be causing conflicts. To test this, could you please report the output of which conda and of which activate? Thanks

sekhwal commented 4 years ago

Hi, Here is the information... $ which conda /home/kumarm/miniconda3/bin/conda $ which activate /home/kumarm/miniconda3/bin/activate (base) kumarm@microbiome:~$

MaestSi commented 4 years ago

Ok, so I guess that if you typels -l /home/kumarm/miniconda3/envs you get MetONTIIME_env but you don't get entrez_qiime_env, right? Please, could you tell me what happens if you type conda create -n entrez_qiime_env python=2.7 gcc? That should be the instruction in the Import_database.sh script that is causing issues. Searching on the net, I found out that yours is a frequent conda error. As described here, the issue might be due to the fact that you installed miniconda3 using sudo. What owner do you get as a result of the ls command, your username or root? If the lattest is the case, a proposed solution would be to write: sudo chown -R username /path/to/miniconda3, of course replacing "username" with your username and the /path/to/miniconda3 with the corresponding path. What still I don't understand is why you got the

NotWritableError: The current user does not have write permissions to a required path. path: /anaconda3/envs/.conda_envs_dir_test

error, that references anaconda3. Simone

sekhwal commented 4 years ago

Hi, I have tried following commands....please take a look..

~$ ls -l /home/kumarm/miniconda3/envs total 4 drwxrwxr-x 26 kumarm kumarm 4096 Jan 3 14:44 MetONTIIME_env ~$ conda create -n entrez_qiime_env python=2.7 gcc

NotWritableError: The current user does not have write permissions to a required path. path: /anaconda3/envs/.conda_envs_dir_test uid: 1024 gid: 1023

If you feel that permissions on this path are set incorrectly, you can manually change them by executing

$ sudo chown 1024:1023 /anaconda3/envs/.conda_envs_dir_test

In general, it's not advisable to use 'sudo conda'.

~$ sudo chown -R username /path/to/miniconda3 [sudo] password for kumarm: kumarm is not in the sudoers file. This incident will be reported.

sekhwal commented 4 years ago

Would you suggest uninstall miniconda3 and reinstall..?

MaestSi commented 4 years ago

Thank you for the information. What is not clear to me is how it has been possible to create one environment (MetONTIIME_env), but it looks like the location where new environments (e.g. entrez_qiime_env) are stored is somehow changed. What happens if you tell conda where to create the new environment? Please try with: conda create -p /home/kumarm/miniconda3/envs/entrez_qiime_env python=2.7 gcc

sekhwal commented 4 years ago

please see the output...

~$ conda create -p /home/kumarm/miniconda3/envs/entrez_qiime_env python=2.7 gcc Collecting package metadata (current_repodata.json): done Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

gcc

Current channels:

To search for alternate channels that may provide the conda package you're looking for, navigate to

https://anaconda.org

and use the search bar at the top of the page.

MaestSi commented 4 years ago

Apparently, gcc has been removed from conda-forge channel. This should be solved adding anaconda channel with: conda config --add channels anaconda and then running: conda create -p /home/kumarm/miniconda3/envs/entrez_qiime_env python=2.7 gcc

sekhwal commented 4 years ago

Hi, These commands work without any errors.. Now I am trying ./Import_database.sh sequence.fasta However it is showing following message.. ./Import_database.sh sequence.fasta WARNING: A conda environment already exists at '/home/kumarm/miniconda3/envs/entrez_qiime_env' Remove existing environment (y/[n])?

MaestSi commented 4 years ago

Please answer n and tell me if it ends without any errors.

sekhwal commented 4 years ago

Hi, it works.......Thank you very much for your kind support...

Here is the process complete successfully without any error.......

$ ./Import_database.sh sequence.fasta WARNING: A conda environment already exists at '/home/kumarm/miniconda3/envs/entrez_qiime_env' Remove existing environment (y/[n])? n

CondaSystemExit: Exiting.

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support Collecting numpy Downloading https://files.pythonhosted.org/packages/3a/5f/47e578b3ae79e2624e205445ab77a1848acdaa2929a00eeef6b16eaaeb20/numpy-1.16.6-cp27-cp27mu-manylinux1_x86_64.whl (17.0MB) |████████████████████████████████| 17.0MB 1.2MB/s Installing collected packages: numpy Successfully installed numpy-1.16.6 DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support Processing /home/kumarm/.cache/pip/wheels/00/d6/78/4b98bf36fba9a1766ab94b453dedaef570bf01dbfcb02f585a/cogent-1.9-cp27-cp27mu-linux_x86_64.whl Installing collected packages: cogent Successfully installed cogent-1.9

Imported sequence.fasta as DNAFASTAFormat to /home/kumarm/MinION-data/MetONTIIME/sequence_sequence.qza Imported /home/kumarm/MinION-data/MetONTIIME/sequence_accession_taxonomy.txt as HeaderlessTSVTaxonomyFormat to /home/kumarm/MinION-data/MetONTIIME/sequence_taxonomy.qza $

sekhwal commented 4 years ago

Would you suggest to update python as it is showing in above message...?

MaestSi commented 4 years ago

Good! So, now you have those 2 QIIME2 artifacts, whose full path you have to set in the config file. And then you should be ready to go. No, I wouldn't suggest to update python as, python2 and python3 follow slightly different syntax rules, and it wouldn't be possible to run python2 code with python3 interpreter. The python2 version we are using, 2.7, is the latest available. The problem is that the entrez_qiime utility which we are using for importing the database, was written a few years ago using python2, and so is now a bit out of date, but I wouldn't be able to update it to python3, as I don't know that language. Python2 is the main reason why I had to create a separate virtual environment entrez_qiime_env, however, as long as it works, it should be ok. Let me know if you are able to start the full pipeline. Maybe you could try first with demo data I have uploaded, or subsampling a few fast5 files with appropriate option in configuration file. Simone

sekhwal commented 4 years ago

Sure! I appreciate...

sekhwal commented 4 years ago

Hi, I have RUN pipeline with fastq sequence files. Therefore, pipeline RUNNING with following message for long time. Please let me know how long usually it take to generate results?

$ ./Launch_MinION_mobile_lab.sh fastq_pass/ $ nohup: appending output to 'nohup.out'

MaestSi commented 4 years ago

Hi, you can check the step completed by the pipeline printing nohup.out file. The most time consuming step is the Blast taxonomy assignment, as it is single threaded. The time it takes to accomplish the job depends mainly on the size of the database you are using. With a quite big db, as the 16S bacterial database, classification should take about 5k reads per hour. Smaller dbs should speed up the analysis consistently. Simone

MaestSi commented 4 years ago

Mind the MetONTIIME pipeline is meant to run starting from fast5 files. If you want to start from already demultiplexed fastq.gz sequences, please have a look at the other open issue for proper command line instruction. I will shortly include that information in the README.

sekhwal commented 4 years ago

It is showing some messages in nohup.out file...Please find pdf file of nohup.out.

Welcome to MinION meta.pdf

Found 0 fastq files.

0% 10 20 30 40 50 60 70 80 90 100% |----|----|----|----|----|----|----|----|----|----| Done in 0 ms. Demultiplexing finished at Thu Jan 9 10:13:50 2020

Error in file(con, "r") : cannot open the connection Calls: grep -> readLines -> file In addition: Warning message: In file(con, "r") : cannot open file 'NA': No such file or directory Execution halted

sekhwal commented 4 years ago

Do I need to provide fast5 files to RUN pipeline. I have these files as well.

sekhwal commented 4 years ago

I think I need to RUN pipeline with fast5 files.

MaestSi commented 4 years ago

Please, see my previous message.

sekhwal commented 4 years ago

Hi,

Here total number of reads in my files, please let me know which number should I take for -d and -t .

Number of reads assigned to BC01: 17 Number of reads assigned to BC02: 10 Number of reads assigned to BC03: 37 Number of reads assigned to BC04: 25 Number of reads assigned to BC05: 13 Number of reads assigned to BC06: 32 Number of reads assigned to BC07: 4 Number of reads assigned to BC08: 25999 Number of reads assigned to BC09: 23324 Number of reads assigned to BC10: 26465 Number of reads assigned to BC11: 27 Number of reads assigned to BC12: 61

MaestSi commented 4 years ago

Hi, it looks like you only included samples in the pool with barcodes BC08, BC09 and BC10. Or, at least, these are the only samples for whom you got a good number of reads. So, if you wanted to start the analysis from fast5 files you could have set: BC_int <- c("BC08", "BC09", "BC10") in the config file. However, you can now simply delete BC\<numbers>.fastq.gz files for the remaining 9 samples. Do you confirm you got all the expected results in the analysis folder? Your question about sampling depth seems to be related to the usage of Evaluate_diversity.sh script, which can only be run after previous analyses have completed successfully. The Evaluate_diversity.sh script is kind of an experiment, as with Illumina reads you would use representative sequences, namely sequences obtained after performing clustering or error correction, for building a taxonomy tree and for evaluating diversity metrics. Since with Nanopore reads we can't perform any meaningful clustering due to high error rate, the script subsamples the same number of reads (-d sampling depth) for each sample, considers these to be the representative sequences, and computes all the diversity metrics. I am not sure if this approach is good enough for evaluating diversity, I am open to suggestions. Maybe diversity metrics that also incorporate phylogenetic distance between the features, as UniFrac, could provide a meaningful representation. In my experience, this is a quite time consuming step, so I would recommend starting with few reads (e.g. 1,000) and then maybe increasing it, using as much threads as you have available, which you can specify with -t parameter. Simone

sekhwal commented 4 years ago

Yes, it seems pipeline generating all files in analysis folder...I RUN pipeline with fast5 files ./Launch_MinION_mobile_lab.sh fast5_pass/ command.

please let me know, is my analysis going correct. I need to add all samples in analysis. How I can improve the reads in other samples?

MaestSi commented 4 years ago

Hi, it looks like your analysis completed successfully. You can monitor the stage of the analysis looking at nohup.out or logfile.txt files. Have a look at barplots following Results visualization section in the README. Unfortunately, if you already set disable_porechop_demu_flag <- 1 which is the default value, there is no way to increase the number of reads assigned to other samples. For diagnosis, have a look at pycoQC report and histograms in qc folder. This way, you can see if reads were not assigned to those samples at all (pycoQC pie chart), or if they have been filtered out due to out of range length (read length histograms). Simone

sekhwal commented 4 years ago

Finally I run the command "Evaluate_diversity.sh", but I did not perform delete BC.fastq.gz as you mentioned above...Is it necessary step? However, I got all necessary file as mentioned in the tutorial..Please let me know which file will generate taxonomy barplot as describing in the tutorial...

./Evaluate_diversity.sh -w /home/kumarm/MinION-data/MetONTIIME/ -m /home/kumarm/MinION-data/MetONTIIME/sequence-metadata.tsv -d 1000 -t 1

sekhwal commented 4 years ago

Also there is another step .."Starting analysis from fastq.gz files". Is the next step or another way to RUN pipeline with fastq.gz files?

MaestSi commented 4 years ago

Not deleting samples with less than "sampling depth" reads is in my opinion not a good idea, as you would be performing comparisons using different numbers of representative sequences, not due to different samples complexity but due to insufficient sequencing depth. taxa-bar-plots.qzv and taxa-bar-plots-no-Unassigned.qzv are the files you can visualize, and you already had them in /home/kumarm/MinION-data/MetONTIIME/fast5_pass_analysis folder. Starting analysis from fastq.gz files is just another way to run the pipeline if you are halfway through. Simone

sekhwal commented 4 years ago

Hi, I was trying to re-run the pipeline from starting. However, it is showing following error while running "./Import_database.sh sequence.fasta", please suggest..

./Import_database.sh: line 55: qiime: command not found ./Import_database.sh: line 61: qiime: command not found

MaestSi commented 4 years ago

Hi, as previously observed, it looks like your miniconda is having some issues in activating the proper environments, possibly due to conflicts with anaconda. Remember that there is no need in running again Import_database.sh, unless you changed the database. Before running the pipeline, you should also check that source activate MetONTIIME_env is actually activating the environment; as a consequence of successfull activation, you should see the environment name in the prompt. If that is the case, run conda deactivate to deactivate it before running the pipeline. Again, also have a look at ls -l /home/kumarm/miniconda3/envs, to see if you have both environments available in the proper location.

sekhwal commented 4 years ago

Hi, Yes, entrez_qiime_env and MetONTIIME_env are properly installed..but still it showing error with ..../Import_database.sh sequence.fasta

drwxrwxr-x 13 kumarm kumarm 4096 Jan 11 21:50 entrez_qiime_env drwxrwxr-x 13 kumarm kumarm 4096 Jan 11 22:22 MetONTIIME_env

ERROR:

Requirement already satisfied: cogent in /home/kumarm/miniconda3/envs/entrez_qiime_env/lib/python2.7/site-packages (1.9) ./Import_database.sh: line 55: qiime: command not found ./Import_database.sh: line 61: qiime: command not found

MaestSi commented 4 years ago

At lines 55 and 61 the command qiime is invoked, which should be available in the path if source activate MetONTIIME_env (line 47) succeeded. Are you able to activate the environment, or is it failing?

sekhwal commented 4 years ago

It is showing..

$ source activate MetONTIIME_env Could not find conda environment: MetONTIIME_env You can list all discoverable environments with conda info --envs.

sekhwal commented 4 years ago

$ which conda /usr/bin/anaconda/bin/conda kumarm@microbiome:~/MinION-data/MetONTIIME$ which activate /usr/bin/anaconda/bin/activate

MaestSi commented 4 years ago

That is the issue I was already thinking about a few days ago. Sometimes you have anaconda bin directory first in the PATH, while sometimes you have miniconda3 bin dir first instead. To solve this, before running the pipeline, simply run in the shell: export PATH=/home/kumarm/miniconda3/bin/:$PATH If you prefer, you can write the instruction in the .bashrc file with the following command: echo "export PATH=/home/kumarm/miniconda3/bin/:$PATH" >> /home/kumarm/.bashrc In this way, every time you open a new shell, the .bashrc file is sourced and you should have miniconda3 bin directory before anaconda's. This should solve all your conda related issues, let me know if this is the case! Simone

sekhwal commented 4 years ago

There is no MetONTIIME_env directory so I am trying to run following command..but it is showing error..

conda create -p /home/kumarm/miniconda3/envs/MetONTIIME_env/ python=2.7 gcc

sekhwal commented 4 years ago

Collecting package metadata (repodata.json): failed

UnavailableInvalidChannel: The channel is not accessible or is invalid. channel name: MetONTIIME_env channel url: https://conda.anaconda.org/MetONTIIME_env error code: 404

You will need to adjust your conda configuration to proceed. Use conda config --show channels to view your configuration's current state, and use conda config --show-sources to view config file locations.

MaestSi commented 4 years ago

Why are you doing this? You told me you already had /home/kumarm/miniconda3/envs/MetONTIIME_env and /home/kumarm/miniconda3/envs/entrez_qiime_env directories. If this is not the case anymore, then repeat installation after following my advice in the previous post. Otherwise, simply run:

export PATH=/home/kumarm/miniconda3/bin/:$PATH
./Import_database.sh sequence.fasta

sekhwal commented 4 years ago

I am not sure, but I think these both env have been removed..I am trying to run following command but it is showing error. Is there any way to reinstall these both env or do I need to delete miniconda3 directory and re-install?

conda create -p /home/kumarm/miniconda3/envs/entrez_qiime_env python=2.7 gcc

conda create -p /home/kumarm/miniconda3/envs/MetONTIIME_env python=2.7 gcc

MaestSi commented 4 years ago

If those environments have been removed, simply repeat MetONTIIME installation following the README, namely git clone the repo and run the install script. Just remember to echo "export PATH=/home/kumarm/miniconda3/bin/:$PATH" >> /home/kumarm/.bashrc and then open a new shell where you can do all the installation. You don't need to delete miniconda3.

sekhwal commented 4 years ago

Hi, I am trying to install pipeline with ./install.sh but it is showing warning message.. Can you please figure out the issue...

Warning: 'bioconda' already in 'channels' list, moving to the top Warning: 'conda-forge' already in 'channels' list, moving to the top Warning: 'r' already in 'channels' list, moving to the top Warning: 'anaconda' already in 'channels' list, moving to the top

MaestSi / MetONTIIME

Regarding configuration pipeline... #6

MINICONDA DIR

basecaller_dir

NCBI-downloaded sequences (QIIME2 artifact)

Taxonomy of NCBI-downloaded sequences (QIIME2 artifact)

sample-metadata file describing samples metadata; it is created automatically if it doesn't exist

load BioStrings package

path to MetONTIIME.sh

path to subsample fast5

SEQTK

PORECHOP

PYCOQC

NANOFILT

there are no any errors while running ./install.sh

MINICONDA DIR

basecaller_dir

NCBI-downloaded sequences (QIIME2 artifact)

Taxonomy of NCBI-downloaded sequences (QIIME2 artifact)

sample-metadata file describing samples metadata; it is created automatically if it doesn't exist

load BioStrings package

path to MetONTIIME.sh

path to subsample fast5

SEQTK

PORECHOP

PYCOQC

NANOFILT

please see the output...

It is showing some messages in nohup.out file...Please find pdf file of nohup.out.

Hi, I am trying to install pipeline with ./install.sh but it is showing warning message.. Can you please figure out the issue...