search

HCGB-IGTP / XICRA

Small RNAseq pipeline for paired-end reads
MIT License
7 stars 3 forks source link

module 'HCGB' has no attribute 'functions' #5

Closed ANDdna1991 closed 1 year ago

ANDdna1991 commented 3 years ago

Hello!

I'm just trying to use XICRA, but apparently something is not working. I just installed the program and tried to run the example. Here is the error. Any clue to fix this? I am working in a cluster.

Traceback (most recent call last): File "/home/ah2192/miniconda3/envs/XICRA/bin/XICRA", line 10, in import XICRA.modules File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/XICRA/init.py", line 17, in from XICRA import File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/XICRA/modules/init.py", line 13, in from XICRA.modules import File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/XICRA/modules/biotype.py", line 21, in from XICRA.config import set_config File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/XICRA/config/init.py", line 6, in from XICRA.config import File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/XICRA/config/extern_progs.py", line 29, in from HCGB import functions File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/HCGB/init.py", line 6, in from HCGB import File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/HCGB/sampleParser/init.py", line 7, in from HCGB.sampleParser import File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/HCGB/sampleParser/files.py", line 15, in from HCGB import functions File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/HCGB/functions/init.py", line 14, in from HCGB.functions import File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/HCGB/functions/main_functions.py", line 29, in import HCGB.functions.aesthetics_functions as HCGB_aes AttributeError: module 'HCGB' has no attribute 'functions'

Thanks, Andres

JFsanchezherrero commented 3 years ago

Hi Andres,

Thank you very much for giving a try to XICRA.

Please, in order to help you I would need to how did you install XICRA.

Did you follow the recipe for installation?

## clone repo
git clone https://github.com/HCGB-IGTP/XICRA.git

## move to folder
cd XICRA

## create conda environemt
conda env create -n XICRA -f XICRA_pip/devel/conda/environment.yml

## activate
conda activate XICRA

## install latest python code
pip install XICRA

## install missing software
sh XICRA_pip/XICRA/config/software/installer.sh

We will soon try to reduce installation steps and include all commands either in a single script or in a conda recipe.

Best regards Jose

ANDdna1991 commented 3 years ago

Hi Jose ^^

Yes, I followed these instructions.

As I can see, the function is in the folder. So, not really sure what can be the problem. I know a little bit of Python but not that much.

(XICRA) [ah2192@login-e-14 functions]$ pwd /home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/HCGB/functions

(XICRA) [ah2192@login-e-14 functions]$ ls aesthetics_functions.py fasta_functions.py info_functions.py main_functions.py pycache time_functions.py blast_functions.py files_functions.py init.py math_functions.py system_call_functions.py

Best, Andres

ANDdna1991 commented 3 years ago

Thank you so much for the quick response.

Actually, there is something weird here...

!# It imports functions

File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/HCGB/sampleParser/files.py", line 15, in from HCGB import functions

!# I guess here you load all the scripts.

File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/HCGB/functions/init.py", line 14, in from HCGB.functions import *

!# but now, it doesn't recognise functions anymore

File "/home/ah2192/miniconda3/envs/XICRA/lib/python3.6/site-packages/HCGB/functions/main_functions.py", line 29, in import HCGB.functions.aesthetics_functions as HCGB_aes AttributeError: module 'HCGB' has no attribute 'functions'

ANDdna1991 commented 3 years ago

I'm still digging.

It seems the issue is in this init.py (HCGB/init.py)

################### all = [ 'sampleParser', 'functions' ]

from HCGB import * ###################

But sampleParser and functions are folders not scripts. I thought you import functions (.py) not folder, right??

ANDdna1991 commented 3 years ago

Ok, I have solved this. Apparently, the code tries repeatedly to import functions which is a folder. so I have commented all these instructions in main_functions.py , info_functions and the HCGB/init.py. Now, it's only in HCGB/functions/init.py where the functions are imported only once.

|==================================================| | Pipeline Configuration | |==================================================|

--------- Starting Process --------- 07/02/2021, 16:05:03

Check dependencies, modules or third party software and print report... ++++++++++++++++++++ Python: ++++++++++++++++++++ Minimum version (3.6) satisfied: 3.6.13 |Anaconda, Inc.| (default, Feb 23 2021, 21:15:04) [GCC 7.3.0]

++++++++++++++++++++ External dependencies: ++++++++++++++++++++ Software: fastqc.........[ OK ] Software: cutadapt.........[ OK ] Software: multiqc.........[ OK ] Software: Rscript.........[ OK ] Software: java..........[ OK ] Software: python.........[ OK ] Software: perl..........[ OK ] Software: make..........[ OK ] Software: git...........[ OK ] Software: fastq-join.........[ OK ]

**ERROR: Software sRNAbench could not be found.

By the way, I found other issue about sRNAbecnch, could you give me some clue?

thanks, Andres

JFsanchezherrero commented 3 years ago

Hi there,

I was also looking at the error. Thanks for the update

I guess I made a mistake when I used import as. Within the same module HCGB (it is a secondary module we also develop) I usually did for clear code:

import HCGB.functions.aesthetics_functions as HCGB_aes

But I guess it is what you mention, It is doubling the import and causing errors.

This is a late implementation I did a few days ago and I am afraid I did not test it.

I would update the code in HCGB and create further checks to verify it is correct now.

Thank you very much for pointing out the issue (and solution!).

Let me know if you find any others or any comments you may have on XICRA. We are still developing it and working on it.

Thank you Andres

JFsanchezherrero commented 3 years ago

Sorry, forgot to mention sRNAbench issue.

This is a known issue as sRNAbench it is not installed either as conda or pip package. We need to install it manually.

We created a shell script that tries to install it.

Let me check in a minute the HCGB code and I would let you know when update. I would suggest to install from scratch or try to reinstall XICRA and HCGB.

ANDdna1991 commented 3 years ago

Hi Jose,

But is the software functional? I would like to do a general annotation of smallRNA seq dataset in different ncRNA subclasses. Also, I would like to look at the manual, I can't find it.

Thanks, Andres

JFsanchezherrero commented 3 years ago

Hi Andres,

I have updated both HCGB and XICRA in pipy. I have additionally tested installation and example analysis. Everything looks fine for me

I would suggest to create a full new installation from scratch and follow steps in the main website.

## clone repo
git clone https://github.com/HCGB-IGTP/XICRA.git

## move to folder
cd XICRA

## create conda environemt
conda env create -n XICRA -f XICRA_pip/devel/conda/environment.yml

## activate
conda activate XICRA

## install latest python code
pip install XICRA

## install missing software
sh XICRA_pip/XICRA/config/software/installer.sh

The last step, we encourage you to use the shell script to install miraligner and sRNAbench. Unfortunately, sRNAbench is not working properly lately. Some issues related to either the website or the softwrae prevent us from using it.

I have call the script and got errors regarding sRNAbench but miraligner seems fine:

Then, I run as an example the example we provide:

## run XICRA example
ln -s ~/BMC_bioinformatics_paper/simulation/example/reads/

## prepare reads
XICRA prep --input reads/ --output_folder test_XICRA

## join reads
XICRA join --input test_XICRA --noTrim

## create miRNA analysis
XICRA miRNA --input test_XICRA --software miraligner sRNAbench

## explore results
ls test_XICRA/report/

It worked for me successfully.

Please let me know if you any additional issues.

For the instructions, follow the command-line instructions for each module. Type

$ XICRA

for all available modules and

$ XICRA miRNA -h

for details on each module.

I encourage you to follow the next steps using each module:

XICRA prep
XICRA trimm
XICRA join (if paired.end data)
XICRA biotype
XICRA miRNA

So far, we only analyze miRNA at the isomiR or total level. We are working in furhter implementations regarding piRNA and tRNA but we might need some months for fully working solution.

Please come back to use for any further details.

Best regards, Jose

ANDdna1991 commented 3 years ago

Hi,

I'm on that, could you please tell me how do you install miraligner ? I'm checking but it's not very clear.

Thanks Andres

JFsanchezherrero commented 3 years ago

Sorry for the confusion.

It is included within the last step of installation instructions 

## install missing software
sh XICRA_pip/XICRA/config/software/installer.sh
JFsanchezherrero commented 3 years ago

Hi Andres,

I wonder if you finally succeeded in the installation and running the test example or your samples using XICRA.

Please let me know if I can help you, Best regards

Jose

ANDdna1991 commented 3 years ago

Hi Jose,

Unfortunately, I am not able to make this running.

When I follow your instructions:

ln -s ~/BMC_bioinformatics_paper/simulation/example/reads/

!## prepare reads XICRA prep --input reads/ --output_folder test_XICRA

I got this error about the example file: ***ERROR: input folder does not exist or it is not readable

|==================================================| | Preparing samples | |==================================================|

--------- Starting Process --------- 07/06/2021, 07:47:09

***ERROR: input folder does not exist or it is not readable

!# Then, if I download directly from GitHub. this run:

XICRA prep --input reads/ --output_folder test_XICRA

but it gives me an error when I run this: XICRA join --input test_XICRA --noTrim

ERROR b'' ERROR

Checking, it appears that the fq files have been joined. However, there is an error: Rep_1_R1.fq.gz doesn't appear to be a fastq file (?).

|==================================================| | Join paired-end reads | |==================================================|

--------- Starting Process --------- 07/06/2021, 07:49:47

gzip: /rds/project/rds-XUr6B1Jhndg/ah2192_backup/projects/small_rna_seq/XICRA/test_XICRA/data/rep_1/raw/rep_1_R2.fq.gz: not in gzip format

gzip: /rds/project/rds-XUr6B1Jhndg/ah2192_backup/projects/small_rna_seq/XICRA/test_XICRA/data/rep_1/raw/rep_1_R1.fq.gz: not in gzip format /rds/project/rds-XUr6B1Jhndg/ah2192_backup/projects/small_rna_seq/XICRA/test_XICRA/data/rep_1/raw/rep_1_R1.fq.gz doesn't appear to be a fastq file (?) ERROR b'' ERROR

* Finish *****

(Time spent: 0 h 0 min 0 s)

Then, I run:

XICRA miRNA --input test_XICRA --software miraligner sRNAbench

it also produces this error:

***ERROR: <Future at 0x2b84b9346358 state=finished raised IndexError> 'rep_1' generated an exception: list index out of range

######################################################################

XICRA pipeline

Jose F. Sanchez & Lauro Sumoy

Copyright (C) 2019-2021 Lauro Sumoy Lab, IGTP, Spain

######################################################################

|==================================================| | miRNA analysis | |==================================================|

--------- Starting Process --------- 07/06/2021, 07:52:55

* Finish *****

(Time spent: 0 h 0 min 1 s)

And checking the output files, there is nothing.

I just used your example dataset (rep_1_R1.fq.gz and rep_1_R2.fq.gz) and submitted your codes.

Which can be the problem?? Please let me know if you need to know anything else. Thanks, Andres

JFsanchezherrero commented 3 years ago

Hi Andres,

Let's see what is going on here.

I guess there is a problem with the input data for the the first problem: ***ERROR: input folder does not exist or it is not readable.

You mentioned you link the example reads simulated we used in the BMC paper:

$ ln -s ~/BMC_bioinformatics_paper/simulation/example/reads/

And then, you try to run XICRA as: $ XICRA prep --input reads/ --output_folder test_XICRA

I guess you did not link properly the reads, maybe you were not in the XICRA git folder. Are you sure there is a folder 'reads' in the working directory containing up to 10 files: 5 samples paired-end reads?

Then, the following error when you download data from GitHub is because you have to download it in raw mode, otherwise you are downloading it properly. That is why the error mentioned later as: 

gzip: /rds/project/rds-XUr6B1Jhndg/ah2192_backup/projects/small_rna_seq/XICRA/test_XICRA/data/rep_1/raw/rep_1_R1.fq.gz: not in gzip format

... doesn't appear to be a fastq file (?)

The later error, the miRNA module error happens as there is no file either joined or trimmed before.

So, to sort this out, I encourage you to either create a symbolic link properly to the reads folder using the full path to the BMC_bioinformatics_paper/simulation/example/reads we provide as an example.

Or on the other hand, try to download and place elsewhere some examples from the github web using wget:

$ mkdir test_wget_git
$ wget https://github.com/HCGB-IGTP/XICRA/raw/master/BMC_bioinformatics_paper/simulation/example/reads/rep_1_R1.fq.gz
$ wget https://github.com/HCGB-IGTP/XICRA/raw/master/BMC_bioinformatics_paper/simulation/example/reads/rep_1_R2.fq.gz
$ wget https://github.com/HCGB-IGTP/XICRA/raw/master/BMC_bioinformatics_paper/simulation/example/reads/rep_2_R1.fq.gz
$ wget https://github.com/HCGB-IGTP/XICRA/raw/master/BMC_bioinformatics_paper/simulation/example/reads/rep_2_R2.fq.gz
$ mv rep_*gz test_wget_git

Now, try again XICRA, with conda environment activated:

XICRA prep --input test_wget_git --output_folder test_XICRA2
XICRA join --input test_XICRA2 --noTrim
XICRA miRNA --input test_XICRA2 --software miraligner optimir

Remember that sRNAbench is not fully working, it wont work properly. Try miraligner and optimir instead.

Best regards Jose

JFsanchezherrero commented 2 years ago

Hi there,

I wonder if you managed to use XICRA with your data or if you need additional help.

Best regards Jose

ANDdna1991 commented 2 years ago

Hi Jose,

Hope you are well. Unfortunately, I am on other things right now, however, I was unable to run XICRA properly (though, I was setting up different programs at the same time and likely I did not spend the proper time). Probably, I will go back to small RNA-seq in some moment. I will give XICRA another try then and I will let you know. Have you done any improvements to the code?

Best, Andrés

JFsanchezherrero commented 1 year ago

Hi there Andres, Just revisiting this issue, we have recently updated the code, installation and provide examples.

Read the documentation available and ask any questions if necessary,

Best regards