Open JohnUrban opened 1 year ago
JohnUrban
commented
1 year ago I am currently debugging.
The problem seems to be in gmetp.pl at this line:
PrepareGenomeTraining($proc) if 1;(I know this is technically not a Braker problem at this point, but I will keep you updated)
JohnUrban
commented
1 year ago And specifically at this line in PrepareGenomeTraining inside the if statement:
if ( CreateThis("hc_regions.gtf"))
{
## breaks with the following line
system( "$bin/create_regions.pl --hcc $hcc_genes --hcp $hcp_genes --out hc_regions.gtf --margin $margin" )
and die "error on create_regions.pl";
}
BenAawf
commented
1 year ago Hi, @JohnUrban first, thanks to you. I can now use conda instead of singularity container.
I have the same issue but using a singularity image from here: https://hub.docker.com/r/teambraker/braker3
And this is the command with singularity: singularity exec -B ${PWD}:${PWD} ${BRAKER_SIF} braker.pl --genome=scaffolds_vf.EDTA_RM_masked.fa --prot_seq=Aves_taxid_8782_3044547_prot.fasta --bam=rna_sorted.bam --softmasking --workingdir=run2 \ --GENEMARK_PATH=${ETP} --PROTHINT_PATH=${ETP}/gmes/ProtHint/bin --threads 72
The same issue as mentioned before :
WARNING: Detected | in fasta header of file /media/ben/Data2TB/test-p/annotation/BRAKER3/Aves_taxid_8782_3044547_prot.fasta. This may later on cause problems! The pipeline will create a new file without spaces or "|" characters and a genome_header.map file to look up the old and new headers. This message will be suppressed from now on!
#*********
ERROR in file /opt/BRAKER/scripts/braker.pl at line 5484
Failed to execute: /usr/bin/perl /media/ben/Data2TB/test-p/annotation/BRAKER3/GeneMark-ETP/bin/etp_release.pl --cfg /media/ben/Data2TB/test-p/annotation/BRAKER3/run2/GeneMark-ETP/etp_config.yaml --workdir /media/ben/Data2TB/test-p/annotation/BRAKER3/run2/GeneMark-ETP --bam /media/ben/Data2TB/test-p/annotation/BRAKER3/run2/GeneMark-ETP/etp_data/ --cores 72 --softmask 1>/media/ben/Data2TB/test-p/annotation/BRAKER3/run2/errors/GeneMark-ETP.stdout 2>/media/ben/Data2TB/test-p/annotation/BRAKER3/run2/errors/GeneMark-ETP.stderr
Failed to execute: /usr/bin/perl /media/ben/Data2TB/test-p/annotation/BRAKER3/GeneMark-ETP/bin/etp_release.pl --cfg /media/ben/Data2TB/test-p/annotation/BRAKER3/run2/GeneMark-ETP/etp_config.yaml --workdir /media/ben/Data2TB/test-p/annotation/BRAKER3/run2/GeneMark-ETP --bam /media/ben/Data2TB/test-p/annotation/BRAKER3/run2/GeneMark-ETP/etp_data/ --cores 72 --softmask 1>/media/ben/Data2TB/test-p/annotation/BRAKER3/run2/errors/GeneMark-ETP.stdout 2>/media/ben/Data2TB/test-p/annotation/BRAKER3/run2/errors/GeneMark-ETP.stderr
The most common problem is an expired or not present file ~/.gm_key!It is clearly related to the GeneMark-ETB LICENCE, which is, in fact not available for use
KatharinaHoff
commented
1 year ago You can download the license for GeneMark-EP from their webserver, and place it in ~/.gm_key .
However, Lars is still working on updating Braker code.
BenAawf @.***> schrieb am Mo. 27. Feb. 2023 um 10:24:
Hi, @JohnUrban https://github.com/JohnUrban first, thank you. I can now use conda instead of singularity container.
I have the same issue but using a singularity image from here: https://hub.docker.com/r/teambraker/braker3
And this is the command with singularity: singularity exec -B ${PWD}:${PWD} ${BRAKER_SIF} braker.pl --genome=scaffolds_vf.EDTA_RM_masked.fa --prot_seq=Aves_taxid_8782_3044547_prot.fasta --bam=rna_sorted.bam --softmasking --workingdir=run2 \ --GENEMARK_PATH=${ETP} --PROTHINT_PATH=${ETP}/gmes/ProtHint/bin --threads 72
The same issue as mentioned before :
WARNING: Detected | in fasta header of file /media/ben/Data2TB/test-p/annotation/BRAKER3/Aves_taxid_8782_3044547_prot.fasta. This may later on cause problems! The pipeline will create a new file without spaces or "|" characters and a genome_header.map file to look up the old and new headers. This message will be suppressed from now on!
*****
ERROR in file /opt/BRAKER/scripts/braker.pl at line 5484 Failed to execute: /usr/bin/perl /media/ben/Data2TB/test-p/annotation/BRAKER3/GeneMark-ETP/bin/etp_release.pl --cfg /media/ben/Data2TB/test-p/annotation/BRAKER3/run2/GeneMark-ETP/etp_config.yaml --workdir /media/ben/Data2TB/test-p/annotation/BRAKER3/run2/GeneMark-ETP --bam /media/ben/Data2TB/test-p/annotation/BRAKER3/run2/GeneMark-ETP/etp_data/ --cores 72 --softmask 1>/media/ben/Data2TB/test-p/annotation/BRAKER3/run2/errors/GeneMark-ETP.stdout 2>/media/ben/Data2TB/test-p/annotation/BRAKER3/run2/errors/GeneMark-ETP.stderr Failed to execute: /usr/bin/perl /media/ben/Data2TB/test-p/annotation/BRAKER3/GeneMark-ETP/bin/etp_release.pl --cfg /media/ben/Data2TB/test-p/annotation/BRAKER3/run2/GeneMark-ETP/etp_config.yaml --workdir /media/ben/Data2TB/test-p/annotation/BRAKER3/run2/GeneMark-ETP --bam /media/ben/Data2TB/test-p/annotation/BRAKER3/run2/GeneMark-ETP/etp_data/ --cores 72 --softmask 1>/media/ben/Data2TB/test-p/annotation/BRAKER3/run2/errors/GeneMark-ETP.stdout 2>/media/ben/Data2TB/test-p/annotation/BRAKER3/run2/errors/GeneMark-ETP.stderr The most common problem is an expired or not present file ~/.gm_key!
It is clearly related to the GeneMark-ETB LICENCE, which is, in fact not available for use
— Reply to this email directly, view it on GitHub https://github.com/Gaius-Augustus/BRAKER/issues/577#issuecomment-1445979978, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJMC6JGBVLEMNGWKYHFTAJTWZRXFFANCNFSM6AAAAAAVHE6PTA . You are receiving this because you are subscribed to this thread.Message ID: @.***>
JohnUrban
commented
1 year ago I have a valid license for GeneMark-ES/ET/EP ver 4.71_lic. It works for both Braker1 and Braker2 runs. It does not work for Braker3 runs. Perhaps that is b/c that license doesn't also work for GeneMark-ETP, which was downloaded separately from: https://github.com/gatech-genemark/GeneMark-ETP .
KatharinaHoff
commented
1 year ago The container has today been updated to contain GeneMark-ETP. You still have to install the license key file in your home directory (license key of GeneMark-ES/ET/EP works) as file ~/.gm_key. Otherwise, it should be very easy to run, now.
JohnUrban
commented
1 year ago Is there a way to get GeneMark-ETP if one is not using the container?
KatharinaHoff
commented
1 year ago Look at the Dockerfile… I think that will answer the question.
John Urban @.***> schrieb am Do. 2. März 2023 um 18:51:
Is there a way to get GeneMark-ETP if one is not using the container?
— Reply to this email directly, view it on GitHub https://github.com/Gaius-Augustus/BRAKER/issues/577#issuecomment-1452277426, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJMC6JHUWFPICZAQDYCG6LLW2DM2BANCNFSM6AAAAAAVHE6PTA . You are receiving this because you commented.Message ID: @.***>
JohnUrban
commented
1 year ago Uh oh - time to learn Docker.
KatharinaHoff
commented
1 year ago No, a keyword search will do for this. There’s a link.
John Urban @.***> schrieb am Do. 2. März 2023 um 18:57:
Uh oh - time to learn Docker.
— Reply to this email directly, view it on GitHub https://github.com/Gaius-Augustus/BRAKER/issues/577#issuecomment-1452291086, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJMC6JDD3Q2BQVE5SVMVWU3W2DNSNANCNFSM6AAAAAAVHE6PTA . You are receiving this because you commented.Message ID: @.***>
JohnUrban
commented
1 year ago Got it! Thanks!
wget http://topaz.gatech.edu/GeneMark/etp.for_braker.tar.gz && \
tar -xzf etp.for_braker.tar.gz && \
mv etp.for_braker ETP && \
chmod a+x /opt/ETP/bin/*py /opt/ETP/bin/*pl /opt/ETP/tools/*Well, I gave Braker3 a try with the GeneMark-ETP copy found here http://topaz.gatech.edu/GeneMark/etp.for_braker.tar.gz -- but gmetp.pl gives the same error that I opened up this thread with -- essentially complete.gtf and complete_uniq.gtf not made/found:
FASTA index file /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/braker3/GeneMark-ETP/data/genome.softmasked.fasta.fai created.
error, file not found: option --f1 complete.gtf
error on open file complete.id: No such file or directory
mv: cannot stat ‘complete_uniq.gtf’: No such file or directory
error on open file /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/braker3/GeneMark-ETP/rnaseq/hints/proteins.fa/complete.gtf: No such file or directory
error on create_regions.pl at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmetp.pl line 2162.I am not using the container, so I know that makes this report extra annoying, and I do apologize for that! I personally couldn't get the whole singularity thing working, it kept complaining about root stuff (I'm on a remote cluster without root/sudo privileges). But if necessary, I will give that route another try in the future -- I'm sure there are local options to learn about.
As for my conda-supported approach detailed in this thread, and especially after getting the same error with the new GeneMark-ETP copy, I am skeptical that this is a ~/.gm_key problem unless I need an entirely new ~/.gm_key for GeneMark-ETP. It works fine for the other GeneMark software with Braker1/Braker2. GeneMark-ETP does run a little bit, but seems to fail to create/find/open those files.
KatharinaHoff
commented
1 year ago The rootless warnings of singularity can be safely ignored.
John Urban @.***> schrieb am Do. 2. März 2023 um 19:26:
Well, I gave Braker3 a try with the GeneMark-ETP copy found here http://topaz.gatech.edu/GeneMark/etp.for_braker.tar.gz -- but gmetp.pl gives the same error that I opened up this thread with -- essentially complete.gtf and complete_uniq.gtf not made/found:
FASTA index file /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/braker3/GeneMark-ETP/data/genome.softmasked.fasta.fai created.
error, file not found: option --f1 complete.gtf
error on open file complete.id: No such file or directory
mv: cannot stat ‘complete_uniq.gtf’: No such file or directory
error on open file /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/braker3/GeneMark-ETP/rnaseq/hints/proteins.fa/complete.gtf: No such file or directory
error on create_regions.pl at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmetp.pl line 2162.
I am not using the container, so I know that makes this report extra annoying, and I do apologize for that! I personally couldn't get the whole singularity thing working, it kept complaining about root stuff (I'm on a remote cluster without root/sudo privileges). But if necessary, I will give that route another try in the future -- I'm sure there are local options to learn about.
As for my conda-supported approach detailed in this thread, and especially after getting the same error with the new GeneMark-ETP copy, I am skeptical that this is a ~/.gm_key problem unless I need an entirely new ~/.gm_key for GeneMark-ETP. It works fine for the other GeneMark software with Braker1/Braker2. GeneMark-ETP does run a little bit, but seems to fail to create/find/open those files.
— Reply to this email directly, view it on GitHub https://github.com/Gaius-Augustus/BRAKER/issues/577#issuecomment-1452341605, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJMC6JHQ5VOCZH5TLU3BWK3W2DQ57ANCNFSM6AAAAAAVHE6PTA . You are receiving this because you commented.Message ID: @.***>
KatharinaHoff
commented
1 year ago Maybe I misunderstood. You very likely need root privileges to install Singularity. If it is not on your cluster, conda is the next best approach.
Did you update Braker? Did you switch to master branch?
Katharina Hoff @.***> schrieb am Do. 2. März 2023 um 19:32:
The rootless warnings of singularity can be safely ignored.
John Urban @.***> schrieb am Do. 2. März 2023 um 19:26:
Well, I gave Braker3 a try with the GeneMark-ETP copy found here http://topaz.gatech.edu/GeneMark/etp.for_braker.tar.gz -- but gmetp.pl gives the same error that I opened up this thread with -- essentially complete.gtf and complete_uniq.gtf not made/found:
FASTA index file /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/braker3/GeneMark-ETP/data/genome.softmasked.fasta.fai created.
error, file not found: option --f1 complete.gtf
error on open file complete.id: No such file or directory
mv: cannot stat ‘complete_uniq.gtf’: No such file or directory
error on open file /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/braker3/GeneMark-ETP/rnaseq/hints/proteins.fa/complete.gtf: No such file or directory
error on create_regions.pl at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmetp.pl line 2162.
I am not using the container, so I know that makes this report extra annoying, and I do apologize for that! I personally couldn't get the whole singularity thing working, it kept complaining about root stuff (I'm on a remote cluster without root/sudo privileges). But if necessary, I will give that route another try in the future -- I'm sure there are local options to learn about.
As for my conda-supported approach detailed in this thread, and especially after getting the same error with the new GeneMark-ETP copy, I am skeptical that this is a ~/.gm_key problem unless I need an entirely new ~/.gm_key for GeneMark-ETP. It works fine for the other GeneMark software with Braker1/Braker2. GeneMark-ETP does run a little bit, but seems to fail to create/find/open those files.
— Reply to this email directly, view it on GitHub https://github.com/Gaius-Augustus/BRAKER/issues/577#issuecomment-1452341605, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJMC6JHQ5VOCZH5TLU3BWK3W2DQ57ANCNFSM6AAAAAAVHE6PTA . You are receiving this because you commented.Message ID: @.***>
JohnUrban
commented
1 year ago Hey - is Braker3 on the master branch now? I was on git checkout braker3.
As for singularity, I installed it using conda. So I have I guess I have a local copy of it. I followed the singularity instructions on the main page (e.g. singularity build braker3.sif docker://teambraker/braker3:latest). It seemed to install, but when I tried the next part (singularity exec braker3.sif print_braker3_setup.py or singularity exec braker3.sif braker.pl), it threw an error. I erased the sif file after that, but at the moment I am re-doing the first step to try to reproduce that error (or get past it).
JohnUrban
commented
1 year ago Ok. For the Singularity errors:
Command:
singularity exec braker3.sif print_braker3_setup.pyError:
INFO: Converting SIF file to temporary sandbox...
FATAL: while extracting braker3.sif: root filesystem extraction failed: extract command failed: ERROR : Failed to create user namespace: user namespace disabled
: exit status 1Other Command:
singularity exec braker3.sif braker.plSame error:
INFO: Converting SIF file to temporary sandbox...
FATAL: while extracting braker3.sif: root filesystem extraction failed: extract command failed: ERROR : Failed to create user namespace: user namespace disabled
: exit status 1
Thamos
commented
1 year ago The container is not working because you didn't install singularity as root. There are possibilities to get it working without root I think, but it depends on the kernel version, see https://docs.sylabs.io/guides/3.5/admin-guide/installation.html#install-nonsetuid The best option would be to ask your cluster admin to install singularity (if possible).
KatharinaHoff
commented
1 year ago I will ask our HPC admin. Possibly fakeroot has to be enabled in Singularity, but I am not sure. I vaguely recall that we discussed that a long time ago.
Edit: Oh, yes, you need to be root to install Singularity.
KatharinaHoff
commented
1 year ago Hey - is Braker3 on the master branch now? I was on
git checkout braker3.
Yes, checkout master. We finally merged. And git pull to update to the latest code.
JohnUrban
commented
1 year ago Ok - I should have mentioned this for the braker3 branch already, but I didn't want to bombard you with issues (more so than I have).
Line 2295 in braker.pl that tries to assess the java version causes an error. The approach to getting the java version changed and the fix is simply changing it back to the old way.
New way:
$cmdString = "java -version 2>&1 | grep \"openjdk version\" | awk -F[\"\.] -v OFS=. '{print \$2,\$3}'";Old way:
$cmdString = "java -version 2>&1 | awk -F[\\\"\\\.] -v OFS=. 'NR==1{print \$2,\$3}'";Full context:
####################### set_JAVA_PATH #######################################
# * set path to java
# * also checks whether java version 1.8 is present
################################################################################
sub set_JAVA_PATH {
my @required_files = ('java');
$JAVA_PATH = set_software_PATH($java_path, "JAVA_PATH",
\@required_files, 'exit');
#$cmdString = "java -version 2>&1 | grep \"openjdk version\" | awk -F[\"\.] -v OFS=. '{print \$2,\$3}'";
$cmdString = "java -version 2>&1 | awk -F[\\\"\\\.] -v OFS=. 'NR==1{print \$2,\$3}'";
my @javav = `$cmdString` or die("Failed to execute: $cmdString");
if(not ($javav[0] =~ m/1\.8/ )){
$prtStr = "\# " . (localtime) . " ERROR: in file " . __FILE__
." at line ". __LINE__ ."\n"
. "You have installed java version $javav[0]. GUSHR requires version 1.8!\n"
. "You can switch between java versions on your system with:\n"
. "sudo update-alternatives --config java\n";
$logString .= $prtStr;
print STDERR $logString;
exit(1);
}
}Alright.
I ran Braker3 again with every thing up-to-date -- noting that I did have to change the java version line back to the old approach as described above.
I still get the same error I opened up this issue with (from braker3/errors/GeneMark-ETP.stderr).
FASTA index file /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/data/genome.softmasked.fasta.fai created.
error, file not found: option --f1 complete.gtf
error on open file complete.id: No such file or directory
mv: cannot stat ‘complete_uniq.gtf’: No such file or directory
error on open file /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/rnaseq/hints/proteins.fa/complete.gtf: No such file or directory
error on create_regions.pl at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmetp.pl line 2162.Here are the contents of the GeneMark-ETP subdirectory inside the Braker3 working directory:
> ls braker3/GeneMark-ETP/
arx data etp_config.yaml etp_data filter_gmst.log proteins.fa prothint_gmst.log rnaseq
> ls braker3/GeneMark-ETP/*/
braker3/GeneMark-ETP/arx/:
chr.names genome.fa
braker3/GeneMark-ETP/data/:
genome.fasta genome.softmasked.fasta genome.softmasked.fasta.fai proteins.fa
braker3/GeneMark-ETP/etp_data/:
forward.bam reverse.bam
braker3/GeneMark-ETP/proteins.fa/:
braker3/GeneMark-ETP/rnaseq/:
gmst hints hisat2 stringtie
> ls braker3/GeneMark-ETP/*/*/
braker3/GeneMark-ETP/rnaseq/gmst/:
GeneMark_hmm.mod genome_gmst_for_HC.gtf genome_gmst.gtf gms.log transcripts_merged.fasta.gff
braker3/GeneMark-ETP/rnaseq/hints/:
bam2hints_forward.gff bam2hints_merged.gff bam2hints_reverse.gff hintsfile_merged.gff proteins.fa
braker3/GeneMark-ETP/rnaseq/hisat2/:
mapping_forward.bam mapping_reverse.bam
braker3/GeneMark-ETP/rnaseq/stringtie/:
transcripts_forward.gff transcripts_merged.fasta transcripts_merged.gff transcripts_reverse.gff
> ls braker3/GeneMark-ETP/*/*/*
braker3/GeneMark-ETP/rnaseq/gmst/GeneMark_hmm.mod braker3/GeneMark-ETP/rnaseq/hints/bam2hints_forward.gff braker3/GeneMark-ETP/rnaseq/hisat2/mapping_reverse.bam
braker3/GeneMark-ETP/rnaseq/gmst/genome_gmst_for_HC.gtf braker3/GeneMark-ETP/rnaseq/hints/bam2hints_merged.gff braker3/GeneMark-ETP/rnaseq/stringtie/transcripts_forward.gff
braker3/GeneMark-ETP/rnaseq/gmst/genome_gmst.gtf braker3/GeneMark-ETP/rnaseq/hints/bam2hints_reverse.gff braker3/GeneMark-ETP/rnaseq/stringtie/transcripts_merged.fasta
braker3/GeneMark-ETP/rnaseq/gmst/gms.log braker3/GeneMark-ETP/rnaseq/hints/hintsfile_merged.gff braker3/GeneMark-ETP/rnaseq/stringtie/transcripts_merged.gff
braker3/GeneMark-ETP/rnaseq/gmst/transcripts_merged.fasta.gff braker3/GeneMark-ETP/rnaseq/hisat2/mapping_forward.bam braker3/GeneMark-ETP/rnaseq/stringtie/transcripts_reverse.gff
braker3/GeneMark-ETP/rnaseq/hints/proteins.fa:
log prothint tmp
...and so onIt cannot find a file located here:
braker3/GeneMark-ETP/rnaseq/hints/proteins.fa/complete.gtf
So I decided to look at what is there:
> ls braker3/GeneMark-ETP/rnaseq/hints/proteins.fa/
log prothint tmpI wonder if the log file (braker3/GeneMark-ETP/rnaseq/hints/proteins.fa/log) holds the answer or a hint - contents:
02-Mar-23 11:36:56 - INFO: Starting the GMST filtering and classification.
02-Mar-23 11:36:56 - INFO: Running the following system call: /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/GeneMarkSTFiltering/gms2hints.pl --tseq /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/rnaseq/stringtie/transcripts_merged.fasta --ggtf /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/rnaseq/stringtie/transcripts_merged.gff --tgff /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/rnaseq/gmst/transcripts_merged.fasta.gff --out /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/rnaseq/hints/proteins.fa/tmp/gmsttbny79f0.gtf
02-Mar-23 11:36:56 - INFO: Making diamond database from /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/data/proteins.fa
02-Mar-23 11:36:56 - INFO: Running the following system call: diamond makedb --in /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/data/proteins.fa -d /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/rnaseq/hints/proteins.fa/tmp/diamondDBbtbbubj0.dmnd
02-Mar-23 11:37:16 - ERROR: Program exited due to an error in command: diamond makedb --in /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/data/proteins.fa -d /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/rnaseq/hints/proteins.fa/tmp/diamondDBbtbbubj0.dmnd
02-Mar-23 11:37:16 - ERROR: Check stderr for more details.When I run the diamond makedb command to see the error:
> diamond makedb --in /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/data/proteins.fa -d /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/rnaseq/hints/proteins.fa/tmp/diamondDBbtbbubj0.dmnd
diamond v2.1.3.157 (C) Max Planck Society for the Advancement of Science
Documentation, support and updates available at http://www.diamondsearch.org
Please cite: http://dx.doi.org/10.1038/s41592-021-01101-x Nature Methods (2021)
#CPU threads: 32
Scoring parameters: (Matrix=BLOSUM62 Lambda=0.267 K=0.041 Penalties=11/1)
Database input file: /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/data/proteins.fa
Opening the database file... [0.143s]
Loading sequences... [3.726s]
Masking sequences... [2.614s]
Writing sequences... [0.562s]
Hashing sequences... [0.239s]
Loading sequences... Error: Error reading input stream at line 7477106: Invalid character (.) in sequenceAnd looking more into that, for some reason many of the OrthoDB protein sequences end with a period... e.g.:
>307491_0:000000
MLAYADNIVVMGETKDINSTSKLISSNNFKYLGVNINNKIGMHIEINERITNGNSCYFSIIKFLRS.I downloaded orthodb proteins like this:
wget --no-check-certificate https://v100.orthodb.org/download/odb10_metazoa_fasta.tar.gz
tar -xzf odb10_metazoa_fasta.tar.gz
cat metazoa/Rawdata/* > metazoan-proteins-orthoDB.fasta
rm -r metazoaI will look into removing these periods.... what strikes me funny though is that this did not cause an error when running Braker2. Or maybe it caused an error that went silent/undetected...?
Thamos
commented
1 year ago I just checked the sequence in orthodb 11 and there it seems to be okay. So maybe if you update to 11 the problem fixes itself.
>307491_0:000000 307491_0
MLAYADNIVVMGETKDINSTSKLISSNNFKYLGVNINNKIGMHIEINERITNGNSCYFSIIKFLRSI re-downloaded the OrthoDB proteins to confirm its not just my copy somehow, and indeed 17014 of the 8266016 metazoan proteins end with a period. But this was still v10 -- thanks to @Thamos for telling me about v11. I didn't realize there had been an update.
Nonetheless, I removed the periods at the end of ODBv10 seqs the following way:
awk '{sub(/\.$/,""); print}' proteins.fa > proteins.fixed.faThat definitely solved the diamond makedb problem. And that in turn might solve my whole issue.... waiting to find out still.
I am still scratching my head as to why Braker2 didn't fail b/c of these period-containing sequences though. I'm somewhat guessing that it might have "failed silently" and perhaps I should be skeptical of those results.
Thamos
commented
1 year ago Do you have a (non empty) "prothint.gff" file in your braker2 directories? I think if "diamond makedb" didn't work there shouldn't be one, as prothint uses diamond. E.g. in my case with orthodb plants it's 54MB.
tomasbruna
commented
1 year ago Hi @JohnUrban, no need to be worried about that. BRAKER2 calls DIAMOND only via ProtHint which sanitizes the protein input (https://github.com/gatech-genemark/ProtHint/commit/19ef04c93bfa691bd6583017189b6e340a4513df).
In BRAKER3, DIAMOND is also called "directly" on raw protein input. That's definitely something to fix, thanks for pointing out.
tomasbruna
commented
1 year ago I'll open an issue about this in GeneMark-ETP.
JohnUrban
commented
1 year ago @Thamos the previous runs with the "dirty" protein sequences did not have that file. Now that I have a "pre-sanitized" protein file, Diamond is happily working at the moment, and I suspect I will get the "prothint.gff" file when it finishes up.
@tomasbruna glad I could point out a real issue here. I will keep you posted on whether or not this allows Braker3 to finish.
JohnUrban
commented
1 year ago @Thamos is there a link like https://v100.orthodb.org/download/odb10_metazoa_fasta.tar.gz for v11?
Else, I could download the whole v11 db here: https://data.orthodb.org/download/ ...do you have any tips on how to filter the entire DB to get just metazoan proteins?
tomasbruna
commented
1 year ago JohnUrban
commented
1 year ago Well, I can confirm that "sanitizing" the protein file allowed Braker3 to get past that step.
The complete.gtf, complete.id, complete_uniq.gtf files are now created/found/openable -- so the main thrust of this issue/thread has been solved.
Nonetheless, a new error arose, and I will continue to diagnose and push through here.
#*********
# WARNING: Number of reliable training genes is low (10). Recommended are at least 600 genes
#*********
ERROR: in file /central/groups/carnegie_poc/jurban/software/braker2/braker3/masterbranch/BRAKER/scripts/./helpMod.pm at line 307
found neither /central/groups/carnegie_poc/jurban/software/conda/anaconda3/envs/braker3-deps2/bin//cfg/tsebra/braker3.cfg nor /central/groups/carnegie_poc/jurban/software/conda/anaconda3/envs/braker3-deps2/scripts//cfg/tsebra/braker3.cfg nor /central/groups/carnegie_poc/jurban/software/conda/anaconda3/envs/braker3-deps2/scripts//cfg/tsebra/braker3.cfg nor /central/groups/carnegie_poc/jurban/software/braker2/braker3/masterbranch/BRAKER/scripts//cfg/tsebra/braker3.cfg!
Please Check the environment variables AUGUSTUS_CONFIG_PATH and command line options AUGUSTUS_BIN_PATH and AUGUSTUS_SCRIPTS_PATH or install AUGUSTUS, again!Note that I am running this on a single Mb "toy sample" to get through all the bugs before launching it on the 500 Mb genome -- hence the warning of a low number of training genes... although the same message for Braker1 was Number of reliable training genes is low (33). and for Braker2 was WARNING: Number of reliable training genes is low (38).... so Braker3 had 3-4-fold fewer training genes at this stage.......
JohnUrban
commented
1 year ago This looks fixable by using a git cloned version of TSEBRA instead of Conda:
> ls ~/software/braker2/associated_software/TSEBRA/config/
braker3.cfg default.cfg keep_ab_initio.cfg pref_braker1.cfgOr actually.... it seems like it is expected to be in BRAKER/scripts/cfg/tsebra but the tsebra subdir is not present :
> ls BRAKER/scripts/cfg/
ep.cfg ep_utr.cfg etp.cfg gth.cfg gth_utr.cfg rnaseq.cfg rnaseq_utr.cfgI just also noticed that braker3/errors/GeneMark-ETP.stderr is full of new errors:
FASTA index file /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/masterbranch/braker3/GeneMark-ETP/data/genome.softmasked.fasta.fai created.
Use of uninitialized value $ph1 in addition (+) at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmes/parse_set.pl line 205.
Use of uninitialized value $ph0 in addition (+) at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmes/parse_set.pl line 205.
Use of uninitialized value $ph2 in addition (+) at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmes/parse_set.pl line 205.
Use of uninitialized value $ph0 in division (/) at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmes/parse_set.pl line 208.
Illegal division by zero at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmes/parse_set.pl line 208.
cat: GT.mat: No such file or directory
cat: AG.mat: No such file or directory
cat: GT.mat: No such file or directory
cat: AG.mat: No such file or directory
cat: GT.mat: No such file or directory
cat: AG.mat: No such file or directory
cat: GC.mat: No such file or directory
cat: GC.mat: No such file or directory
cat: GC.mat: No such file or directory
Illegal division by zero at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/train_super.pl line 184.
error, file not found: option --f1 prothint/prothint.gff
grep: prothint/evidence.gff: No such file or directory
grep: prothint/evidence.gff: No such file or directory
Traceback (most recent call last):
File "/central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/printRnaAlternatives.py", line 353, in <module>
main()
File "/central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/printRnaAlternatives.py", line 289, in main
candidates = loadIntrons(args.genemark)
File "/central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/printRnaAlternatives.py", line 193, in loadIntrons
for row in csv.reader(open(inputFile), delimiter='\t'):
FileNotFoundError: [Errno 2] No such file or directory: 'pred_m/genemark.gtf'
error, file not found: option --f1 prothint/prothint.gff
grep: prothint/evidence.gff: No such file or directory
grep: prothint/evidence.gff: No such file or directory
Died at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/format_back.pl line 14.
Died at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/format_back.pl line 14.
Use of uninitialized value $ph1 in addition (+) at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmes/parse_set.pl line 205.
Use of uninitialized value $ph0 in addition (+) at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmes/parse_set.pl line 205.
Use of uninitialized value $ph2 in addition (+) at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmes/parse_set.pl line 205.
Use of uninitialized value $ph0 in division (/) at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmes/parse_set.pl line 208.
Illegal division by zero at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/gmes/parse_set.pl line 208.
cat: GT.mat: No such file or directory
cat: AG.mat: No such file or directory
cat: GT.mat: No such file or directory
cat: AG.mat: No such file or directory
cat: GT.mat: No such file or directory
cat: AG.mat: No such file or directory
cat: GC.mat: No such file or directory
cat: GC.mat: No such file or directory
cat: GC.mat: No such file or directory
Illegal division by zero at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/train_super.pl line 184.
error, file not found: option --f1 prothint/prothint.gff
grep: prothint/evidence.gff: No such file or directory
grep: prothint/evidence.gff: No such file or directory
Died at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/format_back.pl line 14.
Died at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/alt/ETP/bin/format_back.pl line 14.I'm not sure if these are upstream or downstream of helpMod.pm not finding braker3.cfg.
tomasbruna
commented
1 year ago I've seen these errors when the input is too small and GeneMark does not have enough sequence to train itself on. Did you try testing with our small example at https://github.com/Gaius-Augustus/BRAKER/blob/master/example/tests/test3.sh?
JohnUrban
commented
1 year ago Yes - that is a better idea. I will start using the provided tests. My bad.
Thamos
commented
1 year ago I got the same erros when I tried the new GeneMark-ETP version about two weeks ago (on a whole genome) but thought it's probably because braker wasn't (back then) updatet yet. I'm running it with a full genome currently at the ProtHint step, so we'll see if I also get these erros again.
JohnUrban
commented
1 year ago Alright - so I tested test3.sh.
For it to finish, I need to put the latest github version of TSEBRA in the PATH like this:
TSEBRA=~/software/braker2/associated_software/TSEBRA/
TSEBRA_BIN=${TSEBRA}/bin
TSEBRA_CFG=${TSEBRA}/config
export PATH=${BRAKER3}:${GENEMARK_ETP_BIN}:${GENEMARK_ETP_TOOLS}:${PROTHINT2}:${TSEBRA}:${TSEBRA_BIN}:${TSEBRA_CFG}:${PATH}I'm not totally sure if you need all 3 TSEBRA paths or just to the bin...
When using the tsebra installed as part of Braker3 dependencies with Conda (as described above), it fails with this message:
# WARNING: Number of reliable training genes is low (114). Recommended are at least 600 genes
#*********
ERROR: in file /central/groups/carnegie_poc/jurban/software/braker2/braker3/masterbranch/BRAKER/scripts/./helpMod.pm at line 307
found neither /central/groups/carnegie_poc/jurban/software/conda/anaconda3/envs/braker3-deps2/bin//cfg/tsebra/braker3.cfg nor /central/groups/carnegie_poc/jurban/software/conda/anaconda3/envs/braker3-deps2/scripts//cfg/tsebra/braker3.cfg nor /central/groups/carnegie_poc/jurban/software/conda/anaconda3/envs/braker3-deps2/scripts//cfg/tsebra/braker3.cfg nor /central/groups/carnegie_poc/jurban/software/braker2/braker3/masterbranch/BRAKER/scripts//cfg/tsebra/braker3.cfg!
Please Check the environment variables AUGUSTUS_CONFIG_PATH and command line options AUGUSTUS_BIN_PATH and AUGUSTUS_SCRIPTS_PATH or install AUGUSTUS, again!I do not see the extensive error messages in test3/errors/GeneMark-ETP.stderr as I did with my own sample though.
LarsGab
commented
1 year ago Hi,
the TSEBRA/bin path is enough for BRAKER. Are you sure you are using the latest version of TSEBRA from GitHub?
There should be a configuration file TSEBRA/config/braker3.cfg.
JohnUrban
commented
1 year ago Yes. TSEBRA/bin in the PATH helped solve the braker3.cfg issue. I happened to also include TSEBRA/ and TSEBRA/config in the PATH, but now I have my answer that those latter 2 are not needed.
JohnUrban
commented
1 year ago I have Braker3 working on test3 with UTR=on and my "toy example" (without UTR=on). It is now running on the full genome and all RNA-seq and protein data. I suspect it will finish. When it does, I will report back, and we can close this issue.
KatharinaHoff
commented
1 year ago Please do not run BRAKER3 with UTR=on ... it is not a good idea.
In case of BRAKER3, UTRs could be inferred from the StrinTie assembly. That would be sane to do. It's not smart to use the old GUSHR approach, here. And it's not supported in the container.
JohnUrban
commented
1 year ago Thanks @KatharinaHoff - I have just seen your UTR=on comment. The following is from a UTR=on run from this past weekend. It ended with some weird results that I do not think was from UTR=on, so I will report below.
The Braker3 run did finish on the 500 Mb genome, but the braker3.gtf file did not look right. For example, there are no gene lines. Here is an example:
primary_contig_1 AUGUSTUS stop_codon 102906 102908 . - 0 transcript_id "g4.t1"; gene_id "g4"; supported "False";
primary_contig_1 AUGUSTUS CDS 102906 103304 1 - 0 transcript_id "g4.t1"; gene_id "g4"; cds_type "single";
primary_contig_1 AUGUSTUS start_codon 103302 103304 . - 0 transcript_id "g4.t1"; gene_id "g4"; supported "True";
###
primary_contig_1 AUGUSTUS stop_codon 160846 160848 . - 0 transcript_id "g9.t1"; gene_id "g9"; supported "False";
primary_contig_1 AUGUSTUS CDS 160846 161190 1 - 0 transcript_id "g9.t1"; gene_id "g9"; cds_type "single";
primary_contig_1 AUGUSTUS start_codon 161188 161190 . - 0 transcript_id "g9.t1"; gene_id "g9"; supported "True";
###
primary_contig_1 AUGUSTUS stop_codon 330311 330313 . - 0 transcript_id "g12.t1"; gene_id "g12"; supported "True";
primary_contig_1 AUGUSTUS CDS 330311 330982 0.93 - 0 transcript_id "g12.t1"; gene_id "g12"; cds_type "single";
primary_contig_1 AUGUSTUS start_codon 330980 330982 . - 0 transcript_id "g12.t1"; gene_id "g12"; supported "False";
###
primary_contig_1 AUGUSTUS start_codon 343504 343506 . + 0 transcript_id "g13.t1"; gene_id "g13"; supported "False";
primary_contig_1 AUGUSTUS CDS 343504 344241 1 + 0 transcript_id "g13.t1"; gene_id "g13"; cds_type "single";
primary_contig_1 AUGUSTUS stop_codon 344239 344241 . + 0 transcript_id "g13.t1"; gene_id "g13"; supported "True";
###
primary_contig_1 AUGUSTUS stop_codon 403823 403825 . - 0 transcript_id "g21.t1"; gene_id "g21"; supported "True";
primary_contig_1 AUGUSTUS CDS 403823 404656 1 - 0 transcript_id "g21.t1"; gene_id "g21"; cds_type "single";
primary_contig_1 AUGUSTUS start_codon 404654 404656 . - 0 transcript_id "g21.t1"; gene_id "g21"; supported "True";
###
primary_contig_1 AUGUSTUS stop_codon 414686 414688 . - 0 transcript_id "g23.t1"; gene_id "g23"; supported "False";
primary_contig_1 AUGUSTUS CDS 414686 415387 0.63 - 0 transcript_id "g23.t1"; gene_id "g23"; cds_type "single";
primary_contig_1 AUGUSTUS start_codon 415385 415387 . - 0 transcript_id "g23.t1"; gene_id "g23"; supported "True";
###
primary_contig_1 AUGUSTUS stop_codon 457197 457199 . - 0 transcript_id "g27.t1"; gene_id "g27"; supported "True";
primary_contig_1 AUGUSTUS CDS 457197 457547 0.82 - 0 transcript_id "g27.t1"; gene_id "g27"; cds_type "single";
primary_contig_1 AUGUSTUS start_codon 457545 457547 . - 0 transcript_id "g27.t1"; gene_id "g27"; supported "False";
###
primary_contig_1 AUGUSTUS stop_codon 476245 476247 . - 0 transcript_id "g30.t1"; gene_id "g30"; supported "False";
primary_contig_1 AUGUSTUS CDS 476245 477099 1 - 0 transcript_id "g30.t1"; gene_id "g30"; cds_type "single";The augustus.hints.gtf file looks right though:
primary_contig_97 AUGUSTUS gene 1347 1898 0.57 - . g1
primary_contig_97 AUGUSTUS transcript 1347 1898 0.57 - . g1.t1
primary_contig_97 AUGUSTUS stop_codon 1347 1349 . - 0 transcript_id "g1.t1"; gene_id "g1";
primary_contig_97 AUGUSTUS CDS 1347 1898 0.57 - 0 transcript_id "g1.t1"; gene_id "g1";
primary_contig_97 AUGUSTUS exon 1347 1898 . - . transcript_id "g1.t1"; gene_id "g1";
primary_contig_97 AUGUSTUS start_codon 1896 1898 . - 0 transcript_id "g1.t1"; gene_id "g1";
primary_contig_97 AUGUSTUS gene 18226 18546 1 + . g2
primary_contig_97 AUGUSTUS transcript 18226 18546 1 + . g2.t1
primary_contig_97 AUGUSTUS start_codon 18226 18228 . + 0 transcript_id "g2.t1"; gene_id "g2";
primary_contig_97 AUGUSTUS CDS 18226 18546 1 + 0 transcript_id "g2.t1"; gene_id "g2";
primary_contig_97 AUGUSTUS exon 18226 18546 . + . transcript_id "g2.t1"; gene_id "g2";
primary_contig_97 AUGUSTUS stop_codon 18544 18546 . + 0 transcript_id "g2.t1"; gene_id "g2";
primary_contig_97 AUGUSTUS gene 19143 19775 0.94 + . g3
primary_contig_97 AUGUSTUS transcript 19143 19775 0.94 + . g3.t1
primary_contig_97 AUGUSTUS start_codon 19143 19145 . + 0 transcript_id "g3.t1"; gene_id "g3";
primary_contig_97 AUGUSTUS CDS 19143 19775 0.94 + 0 transcript_id "g3.t1"; gene_id "g3";
primary_contig_97 AUGUSTUS exon 19143 19775 . + . transcript_id "g3.t1"; gene_id "g3";
primary_contig_97 AUGUSTUS stop_codon 19773 19775 . + 0 transcript_id "g3.t1"; gene_id "g3";
primary_contig_97 AUGUSTUS gene 19876 20100 0.99 + . g4
primary_contig_97 AUGUSTUS transcript 19876 20100 0.99 + . g4.t1
primary_contig_97 AUGUSTUS start_codon 19876 19878 . + 0 transcript_id "g4.t1"; gene_id "g4";
primary_contig_97 AUGUSTUS CDS 19876 20100 0.99 + 0 transcript_id "g4.t1"; gene_id "g4";
primary_contig_97 AUGUSTUS exon 19876 20100 . + . transcript_id "g4.t1"; gene_id "g4";
primary_contig_97 AUGUSTUS stop_codon 20098 20100 . + 0 transcript_id "g4.t1"; gene_id "g4";
primary_contig_97 AUGUSTUS gene 20221 22170 0.65 + . g5
primary_contig_97 AUGUSTUS transcript 20221 22170 0.65 + . g5.t1
primary_contig_97 AUGUSTUS start_codon 20221 20223 . + 0 transcript_id "g5.t1"; gene_id "g5";
primary_contig_97 AUGUSTUS CDS 20221 22170 0.65 + 0 transcript_id "g5.t1"; gene_id "g5";
primary_contig_97 AUGUSTUS exon 20221 22170 . + . transcript_id "g5.t1"; gene_id "g5";
primary_contig_97 AUGUSTUS stop_codon 22168 22170 . + 0 transcript_id "g5.t1"; gene_id "g5";This seems similar to the issue reported today:
I will follow some of the debug hints therein, and get back.
JohnUrban
commented
1 year ago RE: UTR=on and using the StringTie assembly to infer UTRs.
Does there exist any tools to use the StringTie assembly to update the GTF with UTR info? Or are we on our own for that for now?
Do you know how a lack of UTRs in transcript sequences affects RNA-seq quantification and differential expression analyses?
JohnUrban
commented
1 year ago @LarsGab thanks for updating the code to remove dots (".") at the end of protein sequences.
I noticed ODB10 and ODB11 also have "*" in tens of thousands of the protein sequences. Does the Braker code also strip those, and if not, could they be causing a problem?
Moreover, ODB10/ODB11 uses an expanded alphabet to represent ambiguous postions. That is to say, in addition to the 20 normal AAs, it also uses some other letters to represent ambiguity: B, Z, X, and J. See the following link for info on those letters: https://www.ddbj.nig.ac.jp/ddbj/code-e.html It also uses "U" for a rare AA (not part of the normal 20): Selenocysteine.
I'd also be curious if these other letters may cause any issues in protein steps.
Overall, I'm trying to diagnose what is causing problems with GeneMark in Braker3 (see #582 as well for example).
Edit: These characters also show up in proteomes downloaded from NCBI.
JohnUrban
commented
1 year ago Update: providing a raw fastq, instead of a STAR produced BAM, solved GeneMark-ETP issues we were having. See #582
JohnUrban
commented
1 year ago Another update:
Cleaning up the protein sequences to have only the 20 canonical amino acid letters did not solve the issue.
JohnUrban
commented
1 year ago Update:
Providing HiSat2 BAMs does not have the error-causing effects of STAR BAMs. Braker3 (and GeneMark) finish fine. So it is specific to STAR BAMs or BAMs produced by aligners other than hisat2.
hisat2 -x ../index/toy -U ../../subsamp.fastq.gz --dta -p 16 | samtools sort --threads 16 > subsam.bamAs far as i remember hisat, by default, produces some sam tags that are needed for stringtie to work. Star doesn't include those tags by default, but I think through some options it can. Did you use those options or did you run it more or less default? Because if default my guess would be that that's the reason for the problems with the star bams.
Edit: Found it again, it's described here in section "Input files". https://ccb.jhu.edu/software/stringtie/index.shtml?t=manual
Hello,
Thank you for all the great tools coming from this team.
I gave Braker3 a shot, but am running into an error at the moment. I will report below how I installed Braker3, and how I used it in case it helps reproduce the error.
I would be grateful for any guidance you can provide, and am eager to get Braker3 working at some point in the near future, but fully understand that you are busy. I am mainly reporting this issue in case it helps your development.
First, here was the command used.
Second, here are the errors as reported.
FASTA index file /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/braker3/GeneMark-ETP/data/genome.softmasked.fasta.fai created. error, file not found: option --f1 complete.gtf error on open file complete.id: No such file or directory mv: cannot stat ‘complete_uniq.gtf’: No such file or directory error on open file /central/groups/carnegie_poc/jurban/data/coral/scratch/toyanno/braker3/braker3/GeneMark-ETP/rnaseq/hints/proteins.fa/complete.gtf: No such file or directory error on create_regions.pl at /central/groups/carnegie_poc/jurban/software/braker2/braker3/deps/genemark-etp/GeneMark-ETP/bin/gmetp.pl line 2162.
mamba env create -f braker3-deps.yml
git clone https://github.com/gatech-genemark/GeneMark-ETP.git
git clone https://github.com/Gaius-Augustus/BRAKER.git cd BRAKER git checkout braker3
conda activate braker3-deps2 export PATH=${BRAKER3}:${GENEMARK_ETP_BIN}:${GENEMARK_ETP_TOOLS}:${PROTHINT2}:${PATH}
name: braker3-deps2 channels: