mutantjoo0 commented 4 years ago

Hello,

I got error while running smeg build_species with example data and my data. I installed smeg as a new environment in conda and additionally installed rename in smeg env. Here I have summarized example dataset and error message with example dataset. #work directory:

(smeg) -bash-4.2$ pwd
/mnt/research/Reguera_Kashefi_Lab/JYL/SMEG/SMEG-1.1.1/test
(smeg) -bash-4.2$ ls
2789STDY5608869.fna  A2-165.fna  AF31-14AC.fna   APC924_119.fna   CNCM_I_4544.fna  mock.fastq.gz  test_db
2789STDY5834970.fna  A2165.fna   APC918_95b.fna  CNCM_I_4541.fna  desman           SL3_3.fna

#command and error message:

(smeg) -bash-4.2$ smeg build_species -g . -o test_db -a -p 16
 ################ Checking for dependencies ########
gcc found
parallel found
R found
Mauve found
roary found
prokka found
bowtie2 found
samtools found
bamtools found
bedtools found
blastn found
All required packages found
 ################ Checking for required R libraries ########
Output directory ok
/mnt/home/leejooy5/miniconda3/envs/smeg/lib/R/bin/exec/R: error while loading shared libraries: libiconv.so.2: cannot open shared object file: No such file or directory
R libraries ok
rm: cannot remove ‘.checkedR’: No such file or directory
build_species option activated
/mnt/research/Reguera_Kashefi_Lab/JYL/SMEG/SMEG-1.1.1/test is present directory
/mnt/ufs18/rs-002/Reguera_Kashefi_Lab/JYL/SMEG/SMEG-1.1.1/test is the genomes directory
/mnt/ufs18/rs-002/Reguera_Kashefi_Lab/JYL/SMEG/SMEG-1.1.1/test/test_db is the output directory
####### PRE PROCESSING GENOMES #####

ERROR: At least, a complete reference genome is required to reorder contigs of draft genomes

Since error message commented about libiconv and I found no libiconv in conda, I installed it in conda.

(smeg) -bash-4.2$ conda list libiconv
# packages in environment at /mnt/home/leejooy5/miniconda3/envs/smeg:
#
# Name                    Version                   Build  Channel
(smeg) -bash-4.2$ conda install libiconv -y -c conda-forge
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /mnt/home/leejooy5/miniconda3/envs/smeg

  added / updated specs:
    - libiconv

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    openssl-1.0.2u             |       h516909a_0         3.2 MB  conda-forge
    ------------------------------------------------------------
                                           Total:         3.2 MB

The following NEW packages will be INSTALLED:

  libiconv           conda-forge/linux-64::libiconv-1.15-h516909a_1006
  python_abi         conda-forge/linux-64::python_abi-3.6-1_cp36m

The following packages will be SUPERSEDED by a higher-priority channel:

  ca-certificates    pkgs/main::ca-certificates-2020.6.24-0 --> conda-forge::ca-certificates-2020.6.20-hecda079_0
  certifi               pkgs/main::certifi-2020.6.20-py36_0 --> conda-forge::certifi-2020.6.20-py36h9f0ad1d_0
  openssl              pkgs/main::openssl-1.0.2u-h7b6447c_0 --> conda-forge::openssl-1.0.2u-h516909a_0

Downloading and Extracting Packages
openssl-1.0.2u       | 3.2 MB    | ################################################################################ | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(smeg) -bash-4.2$ conda list libiconv
# packages in environment at /mnt/home/leejooy5/miniconda3/envs/smeg:
#
# Name                    Version                   Build  Channel
libiconv                  1.15              h516909a_1006    conda-forge

After installing libiconv, I got another error from running smeg build_species as follows:

retry after installing libiconv:

(smeg) -bash-4.2$ smeg build_species -g . -o test_database -a -p 30
 ################ Checking for dependencies ########
gcc found
parallel found
R found
Mauve found
roary found
prokka found
bowtie2 found
samtools found
bamtools found
bedtools found
blastn found
All required packages found
 ################ Checking for required R libraries ########
Output directory ok
/mnt/home/leejooy5/miniconda3/envs/smeg/lib/R/bin/exec/R: error while loading shared libraries: libtinfow.so.6: cannot open shared object file: No such file or directory
R libraries ok
rm: cannot remove ‘.checkedR’: No such file or directory
build_species option activated
/mnt/research/Reguera_Kashefi_Lab/JYL/SMEG/SMEG-1.1.1/test is present directory
/mnt/ufs18/rs-002/Reguera_Kashefi_Lab/JYL/SMEG/SMEG-1.1.1/test is the genomes directory
/mnt/ufs18/rs-002/Reguera_Kashefi_Lab/JYL/SMEG/SMEG-1.1.1/test/test_database is the output directory
####### PRE PROCESSING GENOMES #####

ERROR: At least, a complete reference genome is required to reorder contigs of draft genomes

I tried installing libtinfow, but could not find in conda.

(smeg) -bash-4.2$ conda search libtinfow
Loading channels: done
No match found for: libtinfow. Search: *libtinfow*

PackagesNotFoundError: The following packages are not available from current channels:

  - libtinfow

Current channels:

  - https://repo.anaconda.com/pkgs/main/linux-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-64
  - https://repo.anaconda.com/pkgs/r/noarch
  - https://conda.anaconda.org/conda-forge/linux-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://conda.anaconda.org/bioconda/linux-64
  - https://conda.anaconda.org/bioconda/noarch

I wonder if this error is caused due to uninstalled dependencies or other reason. Thank you for support.

Cheers, Joo-Young

aemiol commented 4 years ago

Hi, Are you using Ubuntu? We have noticed users experience installation problems with Ubuntu. Do you have a means to install SMEG via singularity?

Thanks Tunde

mutantjoo0 commented 4 years ago

Hi Tunde,

Thank you for your support. I am using CentOS Linux (ver. 7) on university HPC. I never used singularity, but I found it is available on HPC system as follows:

(base) -bash-4.2$ singularity --help

Linux container platform optimized for High Performance Computing (HPC) and
Enterprise Performance Computing (EPC)

Usage:
  singularity [global options...]

Description:
  Singularity containers provide an application virtualization layer enabling
  mobility of compute via both application and environment portability. With
  Singularity one is capable of building a root file system that runs on any
  other Linux system where Singularity is installed.

Options:
  -d, --debug     print debugging information (highest verbosity)
  -h, --help      help for singularity
      --nocolor   print without color output (default False)
  -q, --quiet     suppress normal output
  -s, --silent    only print errors
  -v, --verbose   print additional information
      --version   version for singularity

Available Commands:
  build       Build a Singularity image
  cache       Manage the local cache
  capability  Manage Linux capabilities for users and groups
  config      Manage various singularity configuration (root user only)
  delete      Deletes requested image from the library
  exec        Run a command within a container
  help        Help about any command
  inspect     Show metadata for an image
  instance    Manage containers running as services
  key         Manage OpenPGP keys
  oci         Manage OCI containers
  plugin      Manage Singularity plugins
  pull        Pull an image from a URI
  push        Upload image to the provided URI
  remote      Manage singularity remote endpoints
  run         Run the user-defined default command within a container
  run-help    Show the user-defined help for an image
  search      Search a Container Library for images
  shell       Run a shell within a container
  sif         siftool is a program for Singularity Image Format (SIF) file manipulation
  sign        Attach a cryptographic signature to an image
  test        Run the user-defined tests within a container
  verify      Verify cryptographic signatures attached to an image
  version     Show the version for Singularity

Examples:
  $ singularity help <command> [<subcommand>]
  $ singularity help build
  $ singularity help instance start

For additional help or support, please visit https://www.sylabs.io/docs/

I will keep you posted after installation on Singularity and running tutorials with example dataset.

Thanks, Joo-Young

mutantjoo0 commented 4 years ago

Hi Tunde,

I was able to run SMEG through singularity installation. There were so many errors and warnings during process on terminal, but terminal show "DONE" at the end. The followings are outputs from smeg build_species with example data set and my sample.

1)example dataset

(base) -bash-4.2$ singularity exec ~/SMEG/smeg.sif smeg build_species -g . -o test_species_db -a -p 30
 ################ Checking for dependencies ########
gcc found
parallel found
R found
Mauve found
roary found
prokka found
bowtie2 found
samtools found
bamtools found
bedtools found
blastn found
All required packages found
 ################ Checking for required R libraries ########
Output directory ok
R libraries ok
build_species option activated
/mnt/research/Reguera_Kashefi_Lab/JYL/SMEG/SMEG-1.1.1/test is present directory
/mnt/ufs18/rs-002/Reguera_Kashefi_Lab/JYL/SMEG/SMEG-1.1.1/test is the genomes directory
/mnt/ufs18/rs-002/Reguera_Kashefi_Lab/JYL/SMEG/SMEG-1.1.1/test/test_species_db is the output directory
####### PRE PROCESSING GENOMES #####

Complete reference genome(s) detected
####### RE-ORDERING CONTIGS OF DRAFT GENOME(S) (MAUVE)#####

####### ANNOTATING GENOMES (PROKKA) #####

####### CORE-GENOME ANALYSIS (ROARY) #####

Use of uninitialized value in require at /conda/envs/SMEG/lib/site_perl/5.26.2/x86_64-linux-thread-multi/Encode.pm line 61.
.
.
.
rm: cannot remove 'newcoordinates.txt': No such file or directory
rm: cannot remove 'modified_uniq_cluster_SNPs.txt': No such file or directory
wc: F.0.4/*.Input.txt: No such file or directory
wc: F.0.5/*.Input.txt: No such file or directory
wc: F.0.6/*.Input.txt: No such file or directory
wc: F.0.7/*.Input.txt: No such file or directory
wc: F.0.8/*.Input.txt: No such file or directory
wc: F.0.9/*.Input.txt: No such file or directory
wc: T.0.4/*.Input.txt: No such file or directory
wc: T.0.5/*.Input.txt: No such file or directory
wc: T.0.6/*.Input.txt: No such file or directory
wc: T.0.7/*.Input.txt: No such file or directory
wc: T.0.8/*.Input.txt: No such file or directory
wc: T.0.9/*.Input.txt: No such file or directory
####### DONE #####

(base) -bash-4.2$ ls
2789STDY5608869.fna  A2-165.fna  AF31-14AC.fna   APC924_119.fna   CNCM_I_4544.fna  mock.fastq.gz  test_species_db
2789STDY5834970.fna  A2165.fna   APC918_95b.fna  CNCM_I_4541.fna  desman           SL3_3.fna
(base) -bash-4.2$ ls test_species_db/
APC918_95b.fna.fna  F.0.5  F.0.7  F.0.9    reordered_contigs  T.0.5  T.0.7  T.0.9
F.0.4               F.0.6  F.0.8  log.txt  T.0.4              T.0.6  T.0.8  tree.newick

2)my sample data:

(base) -bash-4.2$ singularity exec ~/SMEG/smeg.sif smeg build_species -g . -o Std_db_0812 -a -p 30
 ################ Checking for dependencies ########
gcc found
parallel found
R found
Mauve found
roary found
prokka found
bowtie2 found
samtools found
bamtools found
bedtools found
blastn found
All required packages found
 ################ Checking for required R libraries ########
Output directory ok
R libraries ok
build_species option activated
/mnt/research/Reguera_Kashefi_Lab/JYL/SMEG/input_Std_fa is present directory
/mnt/ufs18/rs-002/Reguera_Kashefi_Lab/JYL/SMEG/input_Std_fa is the genomes directory
/mnt/ufs18/rs-002/Reguera_Kashefi_Lab/JYL/SMEG/input_Std_fa/Std_db_0812 is the output directory
####### PRE PROCESSING GENOMES #####

Complete reference genome(s) detected
####### RE-ORDERING CONTIGS OF DRAFT GENOME(S) (MAUVE)#####

####### ANNOTATING GENOMES (PROKKA) #####

####### CORE-GENOME ANALYSIS (ROARY) #####

Use of uninitialized value in require at /conda/envs/SMEG/lib/site_perl/5.26.2/x86_64-linux-thread-multi/Encode.pm line 61.
.
.
.
Execution halted
rm: cannot remove 'newcoordinates.txt': No such file or directory
rm: cannot remove 'modified_uniq_cluster_SNPs.txt': No such file or directory
wc: F.0.4/*.Input.txt: No such file or directory
wc: F.0.5/*.Input.txt: No such file or directory
wc: F.0.6/*.Input.txt: No such file or directory
wc: F.0.7/*.Input.txt: No such file or directory
wc: F.0.8/*.Input.txt: No such file or directory
wc: F.0.9/*.Input.txt: No such file or directory
wc: T.0.4/*.Input.txt: No such file or directory
wc: T.0.5/*.Input.txt: No such file or directory
wc: T.0.6/*.Input.txt: No such file or directory
wc: T.0.7/*.Input.txt: No such file or directory
wc: T.0.8/*.Input.txt: No such file or directory
wc: T.0.9/*.Input.txt: No such file or directory
####### DONE #####

(base) -bash-4.2$ ls Std_db_0812/
F.0.4  F.0.6  F.0.8  GCA_000006765.1_ASM676v1_genomic.fna.fna  reordered_contigs  T.0.5  T.0.7  T.0.9
F.0.5  F.0.7  F.0.9  log.txt

I guess it is ready to run smeg growth_est with my sample data, am I correct?

ohlab / SMEG

ERROR: At least, a complete reference genome is required to reorder contigs of draft genomes #9

retry after installing libiconv: