search

bedapub / splicekit

splicekit: an integrative toolkit for splicing analysis from short-read RNA-seq
GNU General Public License v3.0
15 stars 4 forks source link

Splicekit.junctions.tab doesn't exist #2

Closed sonicwang1997 closed 1 year ago

sonicwang1997 commented 1 year ago

Hi Greor,

I am running splicekit process, it reminds me of sh: 1: Syntax error: "(" unexpected sh: 1: bsub: not found Traceback (most recent call last): FileNotFoundError: [Errno 2] No such file or directory: 'reference/junctions.tab'. when I check the code of anchors.py, it says open junctions.tab and also other files, but nothing exists in the reference folder.

Thank you for your help.

grexor commented 1 year ago

Dear sonicwang1997,

Thank you for trying out splicekit!

The error above is because splicekit tries to run bsub (expected on a cluster system, otherwise bsub: file not found), please change the config parameter platform to desktop in the splicekit.config file like so:

platform = "desktop"

Hope this helps and welcome to reach out in case of more issues,

Cheers, Gregor

sonicwang1997 commented 1 year ago

Dear Greor,

Many thanks to you for your quick reply. According to your suggestion, it helps me to solve the issue. However, new issues come to me, as the following: [setup] Successfully setup splicekit v0.3 analysis in /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test

sh: 1: source: not found [junctions] reading /home/cera/genomes/homo_sapiens.annotation.None/homo_sapiens.gtf.gz to get gene and exon coordinates processing data/sample_junctions_data/sample_S13_raw.tab Traceback (most recent call last): FileNotFoundError: [Errno 2] No such file or directory: 'data/sample_junctions_data/sample_S13_raw.tab'.

Appreciate all.

Best, Sonic

grexor commented 1 year ago

Could you provide your samples.tab and splicekit.config? If you run source on the command line, does it work? Thanks

sonicwang1997 commented 1 year ago

Dear Greor,

Thank you! here is my samples.tab: sample_id treatment_id S13 AG1.3 S14 AG2.1

here is my samples.config: study_name = "ULB_WB" library_type = "paired-end" # ["paired-end", "single-end"] library_strand = "FIRST_READ_TRANSCRIPTION_STRAND" # ["SECOND_READ_TRANSCRIPTION_STRAND", "FIRST_READ_TRANSCRIPTION_STRAND", "SINGLE_STRAND", "SINGLE_REVERSE", "NONE"] "{{ cookiecutter.rna_library_strand }}"

edgeR result filtering

edgeR_FDR_thr = 0.05

sample annotation (read from samples.tab)

sample_column = "sample_id" treatment_column = "treatment_id" control_name = "AG1.3" separate_column = "" group_column = ""

genome

species = "homo sapien"

genome_version = 38 # pybio genome version, leave None to take latest Ensembl version

bam_path = "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/bam/" gtf_path="/home/cera/genomes/homo_sapiens.annotation.None/" fasta_path="/home/cera/genomes/homo_sapiens.annotation.None/"

shorten comparison names

short_names = []

scanRBP

scanRBP = False # run scanRBP on this dataset protein = "K562.TARDBP.0" protein_label = "tdp43" # used in file names and titles

platform and container

platform = "desktop" # "desktop" or "cluster" (HPC with SLURM) container = "" # leave empty or for singularity use: "singularity run singularity/splicekit.sif"

JBrowse2

jbrowse2_url = ""

I put them in the same folder and then run splicekit process.

Thank you!

grexor commented 1 year ago

If you do:

ls /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/bam/

Do you see the bam files?

Are you on a Unix machine? If you do source on the command line, does it work?

sonicwang1997 commented 1 year ago

Dear Greor,

Yes, I see the bam files: (base) cera@cera-Precision-Tower-5810:~/miniconda3/lib/python3.10/site-packages/splicekit/test$ ls /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/bam/ S13.bam S14.bam S15.bam (base) cera@cera-Precision-Tower-5810:~/miniconda3/lib/python3.10/site-packages/splicekit/test$ source bash: source: filename argument required source: usage: source filename [arguments] I am on a Unix machine. I am not sure if source works.

grexor commented 1 year ago

I would say some of the software is missing (hence sh: 1: source: not found), could you try to run on the command line (in the splicekit folder):

source jobs/jobs_junctions/process.sh

sonicwang1997 commented 1 year ago

Dear Greor,

I am sorry I have no idea about what happened: (base) cera@cera-Precision-Tower-5810:~/miniconda3/lib/python3.10/site-packages/splicekit/test$ source jobs/jobs_junctions/process.sh [splicekit] loading splicekit.config [splicekit] loading splicekit.core [splicekit] loading splicekit.core.annotation [splicekit] loading splicekit.core.features [splicekit] loading splicekit.core.exons [splicekit] loading splicekit.core.genes [splicekit] loading splicekit.core.report [splicekit] loading splicekit.core.patterns [splicekit] loading splicekit.core.promisc [splicekit] loading splicekit.core.anchors [splicekit] loading splicekit.core.junctions [splicekit] loading splicekit.core.jbrowse2_create [splicekit] loading splicekit.core.jbrowse2 [splicekit] loading splicekit.core.juan [splicekit] loading splicekit.judge [splicekit] loading splicekit.core.delta_dar [splicekit] loading splicekit.clusterlogfc [junctions] parsing sam /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/bam//S13.bam to data/sample_junctions_data/sample_S13 [E::idx_find_and_load] Could not retrieve index file for '/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/bam//S13.bam' Traceback (most recent call last): File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/core/junctions.py", line 397, in parse_sam(sys.argv[1], sys.argv[2]) File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/core/junctions.py", line 165, in parse_sam for read in samfile.fetch(): File "pysam/libcalignmentfile.pyx", line 1098, in pysam.libcalignmentfile.AlignmentFile.fetch ValueError: fetch called on bamfile without index [splicekit] loading splicekit.config [splicekit] loading splicekit.core [splicekit] loading splicekit.core.annotation [splicekit] loading splicekit.core.features [splicekit] loading splicekit.core.exons [splicekit] loading splicekit.core.genes [splicekit] loading splicekit.core.report [splicekit] loading splicekit.core.patterns [splicekit] loading splicekit.core.promisc [splicekit] loading splicekit.core.anchors [splicekit] loading splicekit.core.junctions [splicekit] loading splicekit.core.jbrowse2_create [splicekit] loading splicekit.core.jbrowse2 [splicekit] loading splicekit.core.juan [splicekit] loading splicekit.judge [splicekit] loading splicekit.core.delta_dar [splicekit] loading splicekit.clusterlogfc [junctions] parsing sam /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/bam//S14.bam to data/sample_junctions_data/sample_S14 [E::idx_find_and_load] Could not retrieve index file for '/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/bam//S14.bam' Traceback (most recent call last): File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/core/junctions.py", line 397, in parse_sam(sys.argv[1], sys.argv[2]) File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/core/junctions.py", line 165, in parse_sam for read in samfile.fetch(): File "pysam/libcalignmentfile.pyx", line 1098, in pysam.libcalignmentfile.AlignmentFile.fetch ValueError: fetch called on bamfile without index

grexor commented 1 year ago

Looks like the index for bam files is missing, you could try to create it in the bam folder with:

samtools index *.bam

sonicwang1997 commented 1 year ago

Hi Greor,

Sorry for my late reply. After creating the index for bam, I try to run source jobs/jobs_junctions/process.sh. It is OK.

However, when I try to run splicekit process, something might change but also reminding me of the sample_S13.tab is missing, showing like that: [setup] Successfully setup splicekit v0.3 analysis in /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test

sh: 1: source: not found [junctions] reading /home/cera/genomes/homo_sapiens.annotation.None/homo_sapiens.gtf.gz to get gene and exon coordinates processing data/sample_junctions_data/sample_S13_raw.tab processing data/sample_junctions_data/sample_S31_raw.tab {'unresolved': 418339, 'resolved': 78150, 'annotated': 9536} sh: 1: source: not found sh: 1: source: not found [features] Reading annotation: /home/cera/genomes/homo_sapiens.annotation.None/homo_sapiens.gtf.gz [features] saving first_exons (last nucleotide of first exon of each transcript) [features] Reading junctions and anchors annotation: reference/junctions.tab Traceback (most recent call last): File "/home/cera/miniconda3/bin/splicekit", line 130, in splicekit.process(force=args.force) File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/init.py", line 236, in process features() File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/init.py", line 136, in features anchors() File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/init.py", line 157, in anchors splicekit.core.features.read_anchors("donor") File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/core/features.py", line 163, in readanchors f = open(f"data/sample{anchor_type}_anchorsdata/sample{sample_id}.tab", "rt") FileNotFoundError: [Errno 2] No such file or directory: 'data/sample_donor_anchors_data/sample_S13.tab'

grexor commented 1 year ago

Seems there is some software missing on the system, will update the package to report this better. Meanwhile, could you try to run splicekit process in a bash shell?

$ bash
$ splicekit process

This will probably report more in detail on what is missing.

sonicwang1997 commented 1 year ago

Dear Greor,

Thank you for your reply. It show as the following: (base) cera@cera-Precision-Tower-5810:~$ bash (base) cera@cera-Precision-Tower-5810:~$ /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/splicekit process bash: /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/splicekit: /apps/rocs/2020.08/cascadelake/software/Anaconda3/2021.05/bin/python: bad interpreter: No such file or directory

grexor commented 1 year ago

Sorry about this one, I updated the pip and git versions now (splicekit 0.4.5), please re-install and write here if more problems, thanks!

sonicwang1997 commented 1 year ago

Dear Greor,

Thank you for your reply. I guess that problem has been fixed up. But new error might come out like this: [splicekit] v0.4.5, https://github.com/bedapub/splicekit

[splicekit] loading splicekit.config [splicekit] loading splicekit.core [splicekit] loading splicekit.core.annotation [splicekit] loading splicekit.core.features [splicekit] loading splicekit.core.exons [splicekit] loading splicekit.core.genes [splicekit] loading splicekit.core.report [splicekit] loading splicekit.core.patterns [splicekit] loading splicekit.core.promisc [splicekit] loading splicekit.core.anchors [splicekit] loading splicekit.core.junctions [splicekit] loading splicekit.core.jbrowse2_create [splicekit] loading splicekit.core.jbrowse2 [splicekit] loading splicekit.core.juan [splicekit] loading splicekit.judge [splicekit] loading splicekit.core.delta_dar [splicekit] loading splicekit.clusterlogfc Traceback (most recent call last): File "/home/cera/miniconda3/bin/splicekit", line 130, in splicekit.process(force=args.force) File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/init.py", line 234, in process setup() File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/init.py", line 65, in setup print("[setup] Successfully setup splicekit v{version} analysis in {folder}\n".format(folder=os.getcwd(), version=core.current_version)) AttributeError: module 'splicekit.core' has no attribute 'current_version'

grexor commented 1 year ago

Sorry about the trouble, could you please try now (version 0.4.6), I fixed the version variable problem.

sonicwang1997 commented 1 year ago

Dear Greor,

Thank you for your effort. It seems like something wrong: [setup] Successfully setup splicekit v0.4.6 analysis in /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test

sh: 1: source: not found [junctions] reading /home/cera/genomes/homo_sapiens.annotation.None/homo_sapiens.gtf.gz to get gene and exon coordinates processing data/sample_junctions_data/sample_S13_raw.tab processing data/sample_junctions_data/sample_S31_raw.tab {'unresolved': 418339, 'resolved': 78150, 'annotated': 9536} sh: 1: source: not found sh: 1: source: not found [features] Reading annotation: /home/cera/genomes/homo_sapiens.annotation.None/homo_sapiens.gtf.gz splicekit.features: saving last nucleotide of first/second exon of each transcript splicekit.features: reading junctions and anchors annotation from: reference/junctions.tab Traceback (most recent call last): File "/home/cera/miniconda3/bin/splicekit", line 130, in splicekit.process(force=args.force) File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/init.py", line 236, in process features() File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/init.py", line 136, in features anchors() File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/init.py", line 157, in anchors splicekit.core.features.read_anchors("donor") File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/core/features.py", line 166, in readanchors f = open(f"data/sample{anchor_type}_anchorsdata/sample{sample_id}.tab", "rt") FileNotFoundError: [Errno 2] No such file or directory: 'data/sample_donor_anchors_data/sample_S13.tab'

grexor commented 1 year ago

Could you run it again within bash, like you did before? So we can see which software is missing on your computer / cluster. Thanks for the effort, this is really great!

sonicwang1997 commented 1 year ago

Dear Greor,

Sure, here is the output, it came to the same error: (base) cera@cera-Precision-Tower-5810:~$ bash (base) cera@cera-Precision-Tower-5810:~$ /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/splicekit process bash: /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/splicekit: /apps/rocs/2020.08/cascadelake/software/Anaconda3/2021.05/bin/python: bad interpreter: No such file or directory

grexor commented 1 year ago

Could you make sure the splicekit package was re-installed (0.4.6)? Perhaps re-create the virtual environment? Seems its calling an older version of the scripts...

sonicwang1997 commented 1 year ago

(base) cera@cera-Precision-Tower-5810:~$ conda create -n splicekit python=3.8 Collecting package metadata (current_repodata.json): done Solving environment: done

Package Plan

environment location: /home/cera/miniconda3/envs/splicekit

added / updated specs:

The following NEW packages will be INSTALLED:

_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main _openmp_mutex pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu ca-certificates pkgs/main/linux-64::ca-certificates-2023.05.30-h06a4308_0 ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1 libffi pkgs/main/linux-64::libffi-3.4.4-h6a678d5_0 libgcc-ng pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1 libgomp pkgs/main/linux-64::libgomp-11.2.0-h1234567_1 libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1 ncurses pkgs/main/linux-64::ncurses-6.4-h6a678d5_0 openssl pkgs/main/linux-64::openssl-3.0.9-h7f8727e_0 pip pkgs/main/linux-64::pip-23.1.2-py38h06a4308_0 python pkgs/main/linux-64::python-3.8.16-h955ad1f_4 readline pkgs/main/linux-64::readline-8.2-h5eee18b_0 setuptools pkgs/main/linux-64::setuptools-67.8.0-py38h06a4308_0 sqlite pkgs/main/linux-64::sqlite-3.41.2-h5eee18b_0 tk pkgs/main/linux-64::tk-8.6.12-h1ccaba5_0 wheel pkgs/main/linux-64::wheel-0.38.4-py38h06a4308_0 xz pkgs/main/linux-64::xz-5.4.2-h5eee18b_0 zlib pkgs/main/linux-64::zlib-1.2.13-h5eee18b_0

Proceed ([y]/n)? y

Downloading and Extracting Packages

Preparing transaction: done Verifying transaction: done Executing transaction: done #

To activate this environment, use

#

$ conda activate splicekit

#

To deactivate an active environment, use

#

$ conda deactivate

(base) cera@cera-Precision-Tower-5810:~$ conda activate splicekit (base) cera@cera-Precision-Tower-5810:~$ pip install splicekit --upgrade Requirement already satisfied: splicekit in ./miniconda3/lib/python3.10/site-packages (0.4.6) Requirement already satisfied: numpy in ./miniconda3/lib/python3.10/site-packages (from splicekit) (1.25.0) Requirement already satisfied: requests in ./miniconda3/lib/python3.10/site-packages (from splicekit) (2.28.1) Requirement already satisfied: rangehttpserver in ./miniconda3/lib/python3.10/site-packages (from splicekit) (1.3.3) Requirement already satisfied: bs4 in ./miniconda3/lib/python3.10/site-packages (from splicekit) (0.0.1) Requirement already satisfied: pysam in ./miniconda3/lib/python3.10/site-packages (from splicekit) (0.21.0) Requirement already satisfied: scanRBP in ./miniconda3/lib/python3.10/site-packages (from splicekit) (0.1.5) Requirement already satisfied: pybio in ./miniconda3/lib/python3.10/site-packages (from splicekit) (0.3.9) Requirement already satisfied: psutil in ./miniconda3/lib/python3.10/site-packages (from splicekit) (5.9.5) Requirement already satisfied: beautifulsoup4 in ./miniconda3/lib/python3.10/site-packages (from bs4->splicekit) (4.12.2) Requirement already satisfied: cython in ./miniconda3/lib/python3.10/site-packages (from pysam->splicekit) (0.29.35) Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./miniconda3/lib/python3.10/site-packages (from requests->splicekit) (1.26.14) Requirement already satisfied: certifi>=2017.4.17 in ./miniconda3/lib/python3.10/site-packages (from requests->splicekit) (2023.5.7) Requirement already satisfied: idna<4,>=2.5 in ./miniconda3/lib/python3.10/site-packages (from requests->splicekit) (3.4) Requirement already satisfied: charset-normalizer<3,>=2 in ./miniconda3/lib/python3.10/site-packages (from requests->splicekit) (2.0.4) Requirement already satisfied: biopython in ./miniconda3/lib/python3.10/site-packages (from scanRBP->splicekit) (1.81) Requirement already satisfied: matplotlib in ./miniconda3/lib/python3.10/site-packages (from scanRBP->splicekit) (3.7.1) Requirement already satisfied: seaborn in ./miniconda3/lib/python3.10/site-packages (from scanRBP->splicekit) (0.12.2) Requirement already satisfied: scipy in ./miniconda3/lib/python3.10/site-packages (from scanRBP->splicekit) (1.11.0) Requirement already satisfied: soupsieve>1.2 in ./miniconda3/lib/python3.10/site-packages (from beautifulsoup4->bs4->splicekit) (2.4.1) Requirement already satisfied: fonttools>=4.22.0 in ./miniconda3/lib/python3.10/site-packages (from matplotlib->scanRBP->splicekit) (4.40.0) Requirement already satisfied: pillow>=6.2.0 in ./miniconda3/lib/python3.10/site-packages (from matplotlib->scanRBP->splicekit) (9.5.0) Requirement already satisfied: packaging>=20.0 in ./miniconda3/lib/python3.10/site-packages (from matplotlib->scanRBP->splicekit) (23.1) Requirement already satisfied: cycler>=0.10 in ./miniconda3/lib/python3.10/site-packages (from matplotlib->scanRBP->splicekit) (0.11.0) Requirement already satisfied: python-dateutil>=2.7 in ./miniconda3/lib/python3.10/site-packages (from matplotlib->scanRBP->splicekit) (2.8.2) Requirement already satisfied: contourpy>=1.0.1 in ./miniconda3/lib/python3.10/site-packages (from matplotlib->scanRBP->splicekit) (1.1.0) Requirement already satisfied: kiwisolver>=1.0.1 in ./miniconda3/lib/python3.10/site-packages (from matplotlib->scanRBP->splicekit) (1.4.4) Requirement already satisfied: pyparsing>=2.3.1 in ./miniconda3/lib/python3.10/site-packages (from matplotlib->scanRBP->splicekit) (3.1.0) Requirement already satisfied: pandas>=0.25 in ./miniconda3/lib/python3.10/site-packages (from seaborn->scanRBP->splicekit) (2.0.2) Requirement already satisfied: tzdata>=2022.1 in ./miniconda3/lib/python3.10/site-packages (from pandas>=0.25->seaborn->scanRBP->splicekit) (2023.3) Requirement already satisfied: pytz>=2020.1 in ./miniconda3/lib/python3.10/site-packages (from pandas>=0.25->seaborn->scanRBP->splicekit) (2023.3) Requirement already satisfied: six>=1.5 in ./miniconda3/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib->scanRBP->splicekit) (1.16.0) (base) cera@cera-Precision-Tower-5810:~$ bash (base) cera@cera-Precision-Tower-5810:~$ splicekit process [splicekit] v0.4.6, https://github.com/bedapub/splicekit

Please run splicekit in a folder with splicekit.config present (base) cera@cera-Precision-Tower-5810:~$ /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/splicekit process bash: /home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/splicekit: /apps/rocs/2020.08/cascadelake/software/Anaconda3/2021.05/bin/python: bad interpreter: No such file or directory

grexor commented 1 year ago

Seems that when you run:

splicekit process

It tells you that you should run it in a folder with splicekit.config, and then you run:

/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/test/splicekit

Which seems to be another version of splicekit installed somewhere else on your system.

If you do splicekit --version you should get 0.4.6, and then you would move to your folder with the files splicekit.config and samples.tab present, and run splicekit process there.

Hope this helps, Gregor

sonicwang1997 commented 1 year ago

Dear Greor,

I agree with that. I think it is more clear to see what happened. Appreciate for all!

(base) cera@cera-Precision-Tower-5810:~$ bash (base) cera@cera-Precision-Tower-5810:~$ splicekit process splicekit: command not found (base) cera@cera-Precision-Tower-5810:~$ export PATH="$PATH:~/home/cera/miniconda3/bin/splicekit1/test" (base) cera@cera-Precision-Tower-5810:~$ splicekit process splicekit: command not found (base) cera@cera-Precision-Tower-5810:~$ /home/cera/miniconda3/bin/splicekit1/test/splicekit --version [splicekit] v0.4.6, https://github.com/bedapub/splicekit

(base) cera@cera-Precision-Tower-5810:~$ /home/cera/miniconda3/bin/splicekit1/test/splicekit process [splicekit] v0.4.6, https://github.com/bedapub/splicekit

Please run splicekit in a folder with splicekit.config present (base) cera@cera-Precision-Tower-5810:~$ cd /home/cera/miniconda3/bin/splicekit1/test (base) cera@cera-Precision-Tower-5810:~/miniconda3/bin/splicekit1/test$ ls annotation data jbrowse2 jobs logs reference results samples.tab splicekit splicekit.config (base) cera@cera-Precision-Tower-5810:~/miniconda3/bin/splicekit1/test$ splicekit process splicekit: command not found (base) cera@cera-Precision-Tower-5810:~/miniconda3/bin/splicekit1/test$ /home/cera/miniconda3/bin/splicekit1/test/splicekit process [splicekit] v0.4.6, https://github.com/bedapub/splicekit

[splicekit] loading splicekit.config [splicekit] loading splicekit.core [splicekit] loading splicekit.core.annotation [splicekit] loading splicekit.core.features [splicekit] loading splicekit.core.exons [splicekit] loading splicekit.core.genes [splicekit] loading splicekit.core.report [splicekit] loading splicekit.core.patterns [splicekit] loading splicekit.core.promisc [splicekit] loading splicekit.core.anchors [splicekit] loading splicekit.core.junctions [splicekit] loading splicekit.core.jbrowse2_create [splicekit] loading splicekit.core.jbrowse2 [splicekit] loading splicekit.core.juan [splicekit] loading splicekit.judge [splicekit] loading splicekit.core.delta_dar [splicekit] loading splicekit.clusterlogfc [setup] Successfully setup splicekit v0.4.6 analysis in /home/cera/miniconda3/bin/splicekit1/test

sh: 1: source: not found [junctions] reading /home/cera/genomes/homo_sapiens.annotation.None/homo_sapiens.gtf.gz to get gene and exon coordinates processing data/sample_junctions_data/sample_S13_raw.tab Traceback (most recent call last): File "/home/cera/miniconda3/bin/splicekit1/test/splicekit", line 130, in splicekit.process(force=args.force) File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/init.py", line 236, in process features() File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/init.py", line 135, in features junctions_master() File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/init.py", line 90, in junctions_master splicekit.core.junctions.make_master() File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/core/junctions.py", line 256, in make_master junctions = read_raw() File "/home/cera/miniconda3/lib/python3.10/site-packages/splicekit/core/junctions.py", line 239, in read_raw f = open(raw_fname, "rt") FileNotFoundError: [Errno 2] No such file or directory: 'data/sample_junctions_data/sample_S13_raw.tab' (base) cera@cera-Precision-Tower-5810:~/miniconda3/bin/splicekit1/test$

grexor commented 1 year ago

Thank you, could you please run bash before running splicekit process? Then we could see which software is missing...

sonicwang1997 commented 1 year ago

Sorry, I am a bit confused, is it right?

import-im6.q16: not authorized os' @ error/constitute.c/WriteImage/1037. import-im6.q16: not authorizedsys' @ error/constitute.c/WriteImage/1037. import-im6.q16: not authorized argparse' @ error/constitute.c/WriteImage/1037. /home/cera/miniconda3/bin/splicekit1/test/splicekit: line 7: help_0: command not found /home/cera/miniconda3/bin/splicekit1/test/splicekit: line 38: help_edgeR: command not found /home/cera/miniconda3/bin/splicekit1/test/splicekit: line 50: syntax error near unexpected token(' /home/cera/miniconda3/bin/splicekit1/test/splicekit: line 50: `parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter, add_help=False)'

grexor commented 1 year ago

So what I would suggest is this:

$ bash
$ cd folder_with_test_data
$ splicekit process

That would then show if there is any missing software that splicekit calls. The alternative is to compile the singularity image (see provided definition file in the singularity folder).

sonicwang1997 commented 1 year ago

Hi Greor,

Sorry, I didn't fix the problem last time. Untill today, I tried to run again using all samples. It shows me like that. I tried to run singularity. Apparently, it is not easy to deal with kind of problem. Anyway, Thank you for your support!

(base) bioinf@bioinf ~/miniconda3/lib/python3.11/site-packages/splicekit/process $ splicekit process [splicekit] v0.4.6, https://github.com/bedapub/splicekit

[splicekit] loading splicekit.config /bin/sh: 1: pybio: not found [splicekit] loading splicekit.core [splicekit] loading splicekit.core.annotation [splicekit] loading splicekit.core.features [splicekit] loading splicekit.core.exons [splicekit] loading splicekit.core.genes [splicekit] loading splicekit.core.report [splicekit] loading splicekit.core.patterns [splicekit] loading splicekit.core.promisc [splicekit] loading splicekit.core.anchors [splicekit] loading splicekit.core.junctions [splicekit] loading splicekit.core.jbrowse2_create [splicekit] loading splicekit.core.jbrowse2 [splicekit] loading splicekit.core.juan [splicekit] loading splicekit.judge [splicekit] loading splicekit.core.delta_dar [splicekit] loading splicekit.clusterlogfc [setup] Successfully setup splicekit v0.4.6 analysis in /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/process

sh: 1: source: not found [junctions] reading /home/bioinf/genomes/homo_sapiens.annotation.ensembl109/homo_sapiens.gtf.gz to get gene and exon coordinates processing data/sample_junctions_data/sample_S13.bam_raw.tab Traceback (most recent call last): File "/home/bioinf/miniconda3/bin/splicekit", line 130, in splicekit.process(force=args.force) File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/init.py", line 236, in process features() File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/init.py", line 135, in features junctions_master() File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/init.py", line 90, in junctions_master splicekit.core.junctions.make_master() File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/junctions.py", line 256, in make_master junctions = read_raw() ^^^^^^^^^^ File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/junctions.py", line 239, in read_raw f = open(raw_fname, "rt") ^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'data/sample_junctions_data/sample_S13.bam_raw.tab' (base) bioinf@bioinf ~/miniconda3/lib/python3.11/site-packages/splicekit/process $ bash splicekit process import-im6.q16: not authorized os' @ error/constitute.c/WriteImage/1037. import-im6.q16: not authorizedsys' @ error/constitute.c/WriteImage/1037. import-im6.q16: not authorized argparse' @ error/constitute.c/WriteImage/1037. splicekit: line 7: help_0: command not found splicekit: line 38: help_edgeR: command not found splicekit: line 50: syntax error near unexpected token(' splicekit: line 50: `parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter, add_help=False)'

grexor commented 1 year ago

Hello, thanks for reaching out again, if you look at this line:

sh: 1: source: not found

It probably means some software is missing on your system. What you could do is run bash before running the splicekit command, this would probably show us which software is missing.

Another solution is to compile the singularity image provided with splicekit and use that image (change splicekit.config).

Hope this helps, Gregor

sonicwang1997 commented 1 year ago

Hi Geror,

Thank you for your reply. In fact, when I run: bash and then run splicekit process. It is the same without reminding which part is missing. (base) bioinf@bioinf /root $ bash (base) bioinf@bioinf /root $ cd /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/process (base) bioinf@bioinf ~/miniconda3/lib/python3.11/site-packages/splicekit/process $ splicekit process [splicekit] v0.4.6, https://github.com/bedapub/splicekit

[splicekit] loading splicekit.config /bin/sh: 1: pybio: not found [splicekit] loading splicekit.core [splicekit] loading splicekit.core.annotation [splicekit] loading splicekit.core.features [splicekit] loading splicekit.core.exons [splicekit] loading splicekit.core.genes [splicekit] loading splicekit.core.report [splicekit] loading splicekit.core.patterns [splicekit] loading splicekit.core.promisc [splicekit] loading splicekit.core.anchors [splicekit] loading splicekit.core.junctions [splicekit] loading splicekit.core.jbrowse2_create [splicekit] loading splicekit.core.jbrowse2 [splicekit] loading splicekit.core.juan [splicekit] loading splicekit.judge [splicekit] loading splicekit.core.delta_dar [splicekit] loading splicekit.clusterlogfc [setup] Successfully setup splicekit v0.4.6 analysis in /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/process

sh: 1: source: not found [junctions] reading /home/bioinf/genomes/homo_sapiens.annotation.ensembl109/homo_sapiens.gtf.gz to get gene and exon coordinates processing data/sample_junctions_data/sample_S13.bam_raw.tab Traceback (most recent call last): File "/home/bioinf/miniconda3/bin/splicekit", line 130, in splicekit.process(force=args.force) File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/init.py", line 236, in process features() File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/init.py", line 135, in features junctions_master() File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/init.py", line 90, in junctions_master splicekit.core.junctions.make_master() File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/junctions.py", line 256, in make_master junctions = read_raw() ^^^^^^^^^^ File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/junctions.py", line 239, in read_raw f = open(raw_fname, "rt") ^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: 'data/sample_junctions_data/sample_S13.bam_raw.tab'

Thus I am trying singularity, but I am bit confused about singularity, since when I run this, it always showed like this : root@bioinf:~# singularity build splicekit.sif /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/process/splicekit.def WARNING: Authentication token file not found : Only pulls of public images will succeed INFO: Starting build... FATAL: While performing build: conveyor failed to get: Error reading manifest 20.04 in docker.io/library/ubuntu: manifest unknown: OCI index found, but accept header does not support OCI indexes.

But when I run singularity build splicekit.sif docker-archive://ubuntu_20.04.tar, it succeed. root@bioinf:~# singularity build splicekit.sif docker-archive://ubuntu_20.04.tar WARNING: Authentication token file not found : Only pulls of public images will succeed INFO: Starting build... Getting image source signatures Copying blob sha256:f5bb4f853c84f6579d86f72de5f1d43fc8866443d9a027e2106e3755acefd61c 71.68 MiB / 71.68 MiB [====================================================] 4s Copying config sha256:05ba1039f8307a47e87ab71a217ac21ab241a06e3e5e4ff1df5e9e64a402115c 1.18 KiB / 1.18 KiB [======================================================] 0s Writing manifest to image destination Storing signatures INFO: Creating SIF file... INFO: Build complete: splicekit.sif

It would be nice if you could offer more suggestions about singularity. Thank you!

grexor commented 1 year ago

One thing for now is that it seems your system still uses sh instead of bash, and also pybio seems not to be installed in this virtualenv?

/bin/sh: 1: pybio: not found

Perhaps something to investigate, Cheers, Gregor

sonicwang1997 commented 1 year ago

Appreciate!

It seems like something wrong with my bam file, right? yes pybio is not installed in this virtualenv.

(base) bioinf@bioinf ~/miniconda3/lib/python3.11/site-packages/splicekit/process $ ll /bin/sh lrwxrwxrwx 1 root root 4 Jul 21 12:36 /bin/sh -> bash* (base) bioinf@bioinf ~/miniconda3/lib/python3.11/site-packages/splicekit/process $ bash (base) bioinf@bioinf ~/miniconda3/lib/python3.11/site-packages/splicekit/process $ splicekit process [splicekit] v0.4.6, https://github.com/bedapub/splicekit

[splicekit] loading splicekit.config /bin/sh: pybio: command not found [splicekit] loading splicekit.core [splicekit] loading splicekit.core.annotation [splicekit] loading splicekit.core.features [splicekit] loading splicekit.core.exons [splicekit] loading splicekit.core.genes [splicekit] loading splicekit.core.report [splicekit] loading splicekit.core.patterns [splicekit] loading splicekit.core.promisc [splicekit] loading splicekit.core.anchors [splicekit] loading splicekit.core.junctions [splicekit] loading splicekit.core.jbrowse2_create [splicekit] loading splicekit.core.jbrowse2 [splicekit] loading splicekit.core.juan [splicekit] loading splicekit.judge [splicekit] loading splicekit.core.delta_dar [splicekit] loading splicekit.clusterlogfc [setup] Successfully setup splicekit v0.4.6 analysis in /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/process

[splicekit] loading splicekit.config /bin/sh: pybio: command not found [splicekit] loading splicekit.core [splicekit] loading splicekit.core.annotation [splicekit] loading splicekit.core.features [splicekit] loading splicekit.core.exons [splicekit] loading splicekit.core.genes [splicekit] loading splicekit.core.report [splicekit] loading splicekit.core.patterns [splicekit] loading splicekit.core.promisc [splicekit] loading splicekit.core.anchors [splicekit] loading splicekit.core.junctions [splicekit] loading splicekit.core.jbrowse2_create [splicekit] loading splicekit.core.jbrowse2 [splicekit] loading splicekit.core.juan [splicekit] loading splicekit.judge [splicekit] loading splicekit.core.delta_dar [splicekit] loading splicekit.clusterlogfc [junctions] parsing sam /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/ageing.bam/clean/bam.bai//S13.bam.bam to data/sample_junctions_data/sample_S13.bam [E::hts_open_format] Failed to open file "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/ageing.bam/clean/bam.bai//S13.bam.bam" : No such file or directory Traceback (most recent call last): File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/junctions.py", line 397, in parse_sam(sys.argv[1], sys.argv[2]) File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/junctions.py", line 164, in parse_sam samfile = pysam.AlignmentFile(sam_fname, "rb") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/ageing.bam/clean/bam.bai//S13.bam.bam: No such file or directory [splicekit] loading splicekit.config /bin/sh: pybio: command not found [splicekit] loading splicekit.core [splicekit] loading splicekit.core.annotation [splicekit] loading splicekit.core.features [splicekit] loading splicekit.core.exons [splicekit] loading splicekit.core.genes [splicekit] loading splicekit.core.report [splicekit] loading splicekit.core.patterns [splicekit] loading splicekit.core.promisc [splicekit] loading splicekit.core.anchors [splicekit] loading splicekit.core.junctions [splicekit] loading splicekit.core.jbrowse2_create [splicekit] loading splicekit.core.jbrowse2 [splicekit] loading splicekit.core.juan [splicekit] loading splicekit.judge [splicekit] loading splicekit.core.delta_dar [splicekit] loading splicekit.clusterlogfc [junctions] parsing sam /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/ageing.bam/clean/bam.bai//S14.bam.bam to data/sample_junctions_data/sample_S14.bam [E::hts_open_format] Failed to open file "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/ageing.bam/clean/bam.bai//S14.bam.bam" : No such file or directory Traceback (most recent call last): File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/junctions.py", line 397, in parse_sam(sys.argv[1], sys.argv[2]) File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/junctions.py", line 164, in parse_sam samfile = pysam.AlignmentFile(sam_fname, "rb") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/ageing.bam/clean/bam.bai//S14.bam.bam: No such file or directory [splicekit] loading splicekit.config /bin/sh: pybio: command not found [splicekit] loading splicekit.core [splicekit] loading splicekit.core.annotation [splicekit] loading splicekit.core.features [splicekit] loading splicekit.core.exons [splicekit] loading splicekit.core.genes [splicekit] loading splicekit.core.report [splicekit] loading splicekit.core.patterns [splicekit] loading splicekit.core.promisc [splicekit] loading splicekit.core.anchors [splicekit] loading splicekit.core.junctions [splicekit] loading splicekit.core.jbrowse2_create [splicekit] loading splicekit.core.jbrowse2 [splicekit] loading splicekit.core.juan [splicekit] loading splicekit.judge [splicekit] loading splicekit.core.delta_dar [splicekit] loading splicekit.clusterlogfc [junctions] parsing sam /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/ageing.bam/clean/bam.bai//S15.bam.bam to data/sample_junctions_data/sample_S15.bam [E::hts_open_format] Failed to open file "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/ageing.bam/clean/bam.bai//S15.bam.bam" : No such file or directory Traceback (most recent call last): File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/junctions.py", line 397, in parse_sam(sys.argv[1], sys.argv[2]) File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/junctions.py", line 164, in parse_sam samfile = pysam.AlignmentFile(sam_fname, "rb") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "pysam/libcalignmentfile.pyx", line 747, in pysam.libcalignmentfile.AlignmentFile.cinit File "pysam/libcalignmentfile.pyx", line 946, in pysam.libcalignmentfile.AlignmentFile._open FileNotFoundError: [Errno 2] could not open alignment file /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/ageing.bam/clean/bam.bai//S15.bam.bam: No such file or directory

sonicwang1997 commented 1 year ago

Dear Greor,

I am so sorry. I found the problem with my bam file. It made some progress!

Now it came to another remind:

[junctions] processed 148.4 M alignments from /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/ageing.bam/clean/bam.bai//S41.bam [junctions] reading /home/bioinf/genomes/homo_sapiens.annotation.ensembl109/homo_sapiens.gtf.gz to get gene and exon coordinates processing data/sample_junctions_data/sample_S13_raw.tab processing data/sample_junctions_data/sample_S14_raw.tab processing data/sample_junctions_data/sample_S15_raw.tab processing data/sample_junctions_data/sample_S37_raw.tab processing data/sample_junctions_data/sample_S38_raw.tab processing data/sample_junctions_data/sample_S39_raw.tab processing data/sample_junctions_data/sample_S40_raw.tab processing data/sample_junctions_data/sample_S41_raw.tab {'unresolved': 701454, 'resolved': 157431, 'annotated': 28753} jobs/jobs_donor_anchors/process.sh: line 2: featureCounts: command not found cp: cannot stat 'data/sample_donor_anchors_data/sample_S15.tab': No such file or directory tail: cannot open 'data/sample_donor_anchors_data/sample_S15.tab_temp' for reading: No such file or directory rm: cannot remove 'data/sample_donor_anchors_data/sample_S15.tab_temp': No such file or directory mv: cannot stat 'data/sample_donor_anchors_data/sample_S15.tab.summary': No such file or directory jobs/jobs_donor_anchors/process.sh: line 9: featureCounts: command not found cp: cannot stat 'data/sample_donor_anchors_data/sample_S39.tab': No such file or directory tail: cannot open 'data/sample_donor_anchors_data/sample_S39.tab_temp' for reading: No such file or directory rm: cannot remove 'data/sample_donor_anchors_data/sample_S39.tab_temp': No such file or directory mv: cannot stat 'data/sample_donor_anchors_data/sample_S39.tab.summary': No such file or directory

......

[features] Reading annotation: /home/bioinf/genomes/homo_sapiens.annotation.ensembl109/homo_sapiens.gtf.gz splicekit.features: saving last nucleotide of first/second exon of each transcript splicekit.features: reading junctions and anchors annotation from: reference/junctions.tab [features] Anchors read OK, sample=S13, from=data/sample_donor_anchors_data/sample_S13.tab, (1/8, 12.50% done) [features] Anchors read OK, sample=S14, from=data/sample_donor_anchors_data/sample_S14.tab, (2/8, 25.00% done) [features] Anchors read OK, sample=S15, from=data/sample_donor_anchors_data/sample_S15.tab, (3/8, 37.50% done) [features] Anchors read OK, sample=S37, from=data/sample_donor_anchors_data/sample_S37.tab, (4/8, 50.00% done) [features] Anchors read OK, sample=S38, from=data/sample_donor_anchors_data/sample_S38.tab, (5/8, 62.50% done) [features] Anchors read OK, sample=S39, from=data/sample_donor_anchors_data/sample_S39.tab, (6/8, 75.00% done) [features] Anchors read OK, sample=S40, from=data/sample_donor_anchors_data/sample_S40.tab, (7/8, 87.50% done) [features] Anchors read OK, sample=S41, from=data/sample_donor_anchors_data/sample_S41.tab, (8/8, 100.00% done) [features] Anchors read OK, sample=S13, from=data/sample_acceptor_anchors_data/sample_S13.tab, (1/8, 12.50% done) [features] Anchors read OK, sample=S14, from=data/sample_acceptor_anchors_data/sample_S14.tab, (2/8, 25.00% done) [features] Anchors read OK, sample=S15, from=data/sample_acceptor_anchors_data/sample_S15.tab, (3/8, 37.50% done) [features] Anchors read OK, sample=S37, from=data/sample_acceptor_anchors_data/sample_S37.tab, (4/8, 50.00% done) [features] Anchors read OK, sample=S38, from=data/sample_acceptor_anchors_data/sample_S38.tab, (5/8, 62.50% done) [features] Anchors read OK, sample=S39, from=data/sample_acceptor_anchors_data/sample_S39.tab, (6/8, 75.00% done) [features] Anchors read OK, sample=S40, from=data/sample_acceptor_anchors_data/sample_S40.tab, (7/8, 87.50% done) [features] Anchors read OK, sample=S41, from=data/sample_acceptor_anchors_data/sample_S41.tab, (8/8, 100.00% done) [features] Saved comparison_donor_anchors_data/AG2_AG1_A.tab (1/1, 100.00% done) [features] Saved comparison_acceptor_anchors_data/AG2_AG1_A.tab (1/1, 100.00% done) [core.junctions] reading raw junctions: data/sample_junctions_data/sample_S13_raw.tab [core.junctions] writting junctions counts file: data/sample_junctions_data/sample_S13.tab [core.junctions] reading raw junctions: data/sample_junctions_data/sample_S14_raw.tab [core.junctions] writting junctions counts file: data/sample_junctions_data/sample_S14.tab [core.junctions] reading raw junctions: data/sample_junctions_data/sample_S15_raw.tab [core.junctions] writting junctions counts file: data/sample_junctions_data/sample_S15.tab [core.junctions] reading raw junctions: data/sample_junctions_data/sample_S37_raw.tab [core.junctions] writting junctions counts file: data/sample_junctions_data/sample_S37.tab [core.junctions] reading raw junctions: data/sample_junctions_data/sample_S38_raw.tab [core.junctions] writting junctions counts file: data/sample_junctions_data/sample_S38.tab [core.junctions] reading raw junctions: data/sample_junctions_data/sample_S39_raw.tab [core.junctions] writting junctions counts file: data/sample_junctions_data/sample_S39.tab [core.junctions] reading raw junctions: data/sample_junctions_data/sample_S40_raw.tab [core.junctions] writting junctions counts file: data/sample_junctions_data/sample_S40.tab [core.junctions] reading raw junctions: data/sample_junctions_data/sample_S41_raw.tab [core.junctions] writting junctions counts file: data/sample_junctions_data/sample_S41.tab [features] Reading annotation: /home/bioinf/genomes/homo_sapiens.annotation.ensembl109/homo_sapiens.gtf.gz splicekit.features: saving last nucleotide of first/second exon of each transcript splicekit.features: reading junctions and anchors annotation from: reference/junctions.tab [features] Junction read OK, sample=S13, from=data/sample_junctions_data/sample_S13.tab, (1/8, 12.50% done) [features] Junction read OK, sample=S14, from=data/sample_junctions_data/sample_S14.tab, (2/8, 25.00% done) [features] Junction read OK, sample=S15, from=data/sample_junctions_data/sample_S15.tab, (3/8, 37.50% done) [features] Junction read OK, sample=S37, from=data/sample_junctions_data/sample_S37.tab, (4/8, 50.00% done) [features] Junction read OK, sample=S38, from=data/sample_junctions_data/sample_S38.tab, (5/8, 62.50% done) [features] Junction read OK, sample=S39, from=data/sample_junctions_data/sample_S39.tab, (6/8, 75.00% done) [features] Junction read OK, sample=S40, from=data/sample_junctions_data/sample_S40.tab, (7/8, 87.50% done) [features] Junction read OK, sample=S41, from=data/sample_junctions_data/sample_S41.tab, (8/8, 100.00% done) [features] Saved comparison_junctions_data/AG2_AG1_A.tab (1/1, 100.00% done) [features] reading donor anchors from: data/comparison_donor_anchors_data/AG2_AG1_A.tab [features] reading acceptor anchors from: data/comparison_acceptor_anchors_data/AG2_AG1_A.tab [features] processing junctions from: data/comparison_junctions_data/AG2_AG1_A.tab [features] Added DAI for comparison_junctions_data/AG2_AG1_A.tab (1/1, 100.00% done) jobs/jobs_exons/process.sh: line 2: featureCounts: command not found cp: cannot stat 'data/sample_exons_data/sample_S15.tab': No such file or directory tail: cannot open 'data/sample_exons_data/sample_S15.tab_temp' for reading: No such file or directory rm: cannot remove 'data/sample_exons_data/sample_S15.tab_temp': No such file or directory mv: cannot stat 'data/sample_exons_data/sample_S15.tab.summary': No such file or directory

....... [features] Reading annotation: /home/bioinf/genomes/homo_sapiens.annotation.ensembl109/homo_sapiens.gtf.gz splicekit.features: saving last nucleotide of first/second exon of each transcript splicekit.features: reading junctions and anchors annotation from: reference/junctions.tab [features] Exons read OK, sample=S13, from=data/sample_exons_data/sample_S13.tab, (1/8, 12.50% done) [features] Exons read OK, sample=S14, from=data/sample_exons_data/sample_S14.tab, (2/8, 25.00% done) [features] Exons read OK, sample=S15, from=data/sample_exons_data/sample_S15.tab, (3/8, 37.50% done) [features] Exons read OK, sample=S37, from=data/sample_exons_data/sample_S37.tab, (4/8, 50.00% done) [features] Exons read OK, sample=S38, from=data/sample_exons_data/sample_S38.tab, (5/8, 62.50% done) [features] Exons read OK, sample=S39, from=data/sample_exons_data/sample_S39.tab, (6/8, 75.00% done) [features] Exons read OK, sample=S40, from=data/sample_exons_data/sample_S40.tab, (7/8, 87.50% done) [features] Exons read OK, sample=S41, from=data/sample_exons_data/sample_S41.tab, (8/8, 100.00% done) [features] Saved comparison_exons_data/AG2_AG1_A.tab (1/1, 100.00% done) sh: bsub: command not found jobs/jobs_genes/process.sh: line 2: featureCounts: command not found cp: cannot stat 'data/sample_genes_data/sample_S15.tab': No such file or directory tail: cannot open 'data/sample_genes_data/sample_S15.tab_temp' for reading: No such file or directory rm: cannot remove 'data/sample_genes_data/sample_S15.tab_temp': No such file or directory mv: cannot stat 'data/sample_genes_data/sample_S15.tab.summary': No such file or directory

......

splicekit.features: saving last nucleotide of first/second exon of each transcript splicekit.features: reading junctions and anchors annotation from: reference/junctions.tab [features] Genes read OK, sample=S13, from=data/sample_genes_data/sample_S13.tab, (1/8, 12.50% done) [features] Genes read OK, sample=S14, from=data/sample_genes_data/sample_S14.tab, (2/8, 25.00% done) [features] Genes read OK, sample=S15, from=data/sample_genes_data/sample_S15.tab, (3/8, 37.50% done) [features] Genes read OK, sample=S37, from=data/sample_genes_data/sample_S37.tab, (4/8, 50.00% done) [features] Genes read OK, sample=S38, from=data/sample_genes_data/sample_S38.tab, (5/8, 62.50% done) [features] Genes read OK, sample=S39, from=data/sample_genes_data/sample_S39.tab, (6/8, 75.00% done) [features] Genes read OK, sample=S40, from=data/sample_genes_data/sample_S40.tab, (7/8, 87.50% done) [features] Genes read OK, sample=S41, from=data/sample_genes_data/sample_S41.tab, (8/8, 100.00% done) [features] Saved comparison_genes_data/AG2_AG1_A.tab (1/1, 100.00% done) jobs/jobs_edgeR_junctions/process.sh: line 2: /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/comps_edgeR.R: No such file or directory Generating edgeR results file=results/results_edgeR_junctions.tab, considering all results with FDR<=0.05

[features] Reading annotation: /home/bioinf/genomes/homo_sapiens.annotation.ensembl109/homo_sapiens.gtf.gz splicekit.features: saving last nucleotide of first/second exon of each transcript splicekit.features: reading junctions and anchors annotation from: reference/junctions.tab jobs/jobs_edgeR_exons/process.sh: line 2: /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/comps_edgeR.R: No such file or directory Generating edgeR results file=results/results_edgeR_exons.tab, considering all results with FDR<=0.05

jobs/jobs_edgeR_genes/process.sh: line 2: /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/comps_edgeR.R: No such file or directory Generating edgeR results file=results/results_edgeR_genes.tab, considering all results with FDR<=0.05

jobs/jobs_edgeR_donor_anchors/process.sh: line 2: /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/comps_edgeR.R: No such file or directory Generating edgeR results file=results/results_edgeR_donor_anchors.tab, considering all results with FDR<=0.05

jobs/jobs_edgeR_acceptor_anchors/process.sh: line 2: /home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/comps_edgeR.R: No such file or directory Generating edgeR results file=results/results_edgeR_acceptor_anchors.tab, considering all results with FDR<=0.05

[juan] reading: results/results_edgeR_donor_anchors/AG2_AG1_A_altsplice.tab [juan] warning, do such file: results/results_edgeR_donor_anchors/AG2_AG1_A_altsplice.tab [juan] reading: results/results_edgeR_acceptor_anchors/AG2_AG1_A_altsplice.tab [juan] warning, do such file: results/results_edgeR_acceptor_anchors/AG2_AG1_A_altsplice.tab Traceback (most recent call last): File "/home/bioinf/miniconda3/bin/splicekit", line 130, in splicekit.process(force=args.force) File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/init.py", line 239, in process motifs() File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/init.py", line 215, in motifs import splicekit.core.motifs File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/motifs.py", line 20, in import Levenshtein as lev ModuleNotFoundError: No module named 'Levenshtein'

grexor commented 1 year ago

Yep, seems you are missing the software featureCounts (https://www.rdocumentation.org/packages/Rsubread/versions/1.22.2/topics/featureCounts), hope this helps, Gregor

sonicwang1997 commented 1 year ago

Dear Greor,

Thank you so much! Now it goes well. However, I always get the remind like this, I am wondering if it is because of my data.

Generating edgeR results file=results/results_edgeR_acceptor_anchors.tab, considering all results with FDR<=0.05

Processed results/results_edgeR_acceptor_anchors/AG3_AG1_A_altsplice.tab 1/1 (100.00% done) [juan] reading: results/results_edgeR_donor_anchors/AG3_AG1_A_altsplice.tab [juan] reading: results/results_edgeR_acceptor_anchors/AG3_AG1_A_altsplice.tab [splicekit] [scanRBP] processing junctions criteria [splicekit] [scanRBP] AG3_AG1_A: junctions_up_controls junctions sh: scanRBP: command not found [splicekit] [scanRBP] AG3_AG1_A: junctions_down_controls junctions sh: scanRBP: command not found [splicekit] [scanRBP] processing exons criteria [splicekit] fasta files save [splicekit] fasta files save QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-bioinf' /home/bioinf/miniconda3/lib/python3.11/site-packages/logomaker/src/Logo.py:1062: UserWarning: Attempting to set identical low and high ylims makes transformation singular; automatically expanding. self.ax.set_ylim([ymin, ymax]) Traceback (most recent call last): File "/home/bioinf/miniconda3/bin/splicekit", line 130, in splicekit.process(force=args.force) File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/init.py", line 239, in process motifs() File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/init.py", line 222, in motifs splicekit.core.motifs.process() File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/motifs.py", line 724, in process make_logos() File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/motifs.py", line 539, in make_logos seqs_mean_lev_pariwise_dist = compute_mean_levdist(seq_list=seqs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/bioinf/miniconda3/lib/python3.11/site-packages/splicekit/core/motifs.py", line 93, in compute_mean_levdist seq_dist_avg_norm = seq_dist_avg/seqs_len_avg


ZeroDivisionError: float division by zero
grexor commented 1 year ago

Thanks for pointing this out, will fix these border conditions (for when there are no sequences detected) asap, Cheers, Gregor