search

lfaino / LoReAn

Long Reads Annotation pipeline
MIT License
70 stars 10 forks source link

raise ValueError("No lines parsed -- was an empty file provided?") ValueError: No lines parsed -- was an empty file provided? #16

Open ncgrjsmiao opened 5 years ago

ncgrjsmiao commented 5 years ago

I test the example dataset within singularity, but met some probelm until ###GMAP MAPPING STARTED AT: 13:06:26 21-03 ###

BUILD INDEX

bellow is Error message: Traceback (most recent call last): File "/opt/LoReAn/code/lorean.py", line 552, in main() File "/opt/LoReAn/code/lorean.py", line 497, in main args.verbose, Fflag=True) File "/opt/LoReAn/code/mapping.py", line 398, in gmap wd, Fflag, type_out, verbose) File "/opt/LoReAn/code/mapping.py", line 130, in gmap_map filename = longest_cds(list_fasta[0][1], list_fasta[1][1], verbose, working_dir, filename) File "/opt/LoReAn/code/mapping.py", line 188, in longest_cds db = gffutils.create_db(gff_file, ':memory:', merge_strategy='create_unique', keep_order=True, transform=transform) File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 1288, in create_db c.create() File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 504, in create self._populate_from_lines(self.iterator) File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 625, in _populate_from_lines raise ValueError("No lines parsed -- was an empty file provided?") ValueError: No lines parsed -- was an empty file provided?

lfaino commented 5 years ago

Hi, did you install via docker or singularity?

Cheers Luigi

cgjosephlee commented 5 years ago

Had same problem here on Crispa example. I installed using docker.

$ lorean.py -a -rp repeats.scaffold3.bed -sr scaffold3.short_1.fastq,scaffold3.short_2.fastq  -lr scaffold3.long.fasta  -pr scaffold3.prot.fasta -sp crispa scaffold3.fasta -d -f -mg -t 60 --keep_tmp

 ### LOREAN IS CHECKING THAT GMAP IS CORRECTLY BUILD ###

###GMAP MAPPING  STARTED AT:    08:49:22 01-04  ###
        ###BUILD INDEX###
        ###MAP###

 ### GMAP IS CORRECTLY BUILD ###

###MASKING THE GENOME STARTED AT:       08:49:21 01-04  ###

###STAR MAPPING  STARTED AT:    08:50:05 01-04  ###
        ###BUILD INDEX###
        ###MAP###
        ###SAM to BAM###
        ###SORTING BAM###

###TRINITY STARTS AT:   08:52:36 01-04  ###

###GMAP MAPPING  STARTED AT:    09:02:09 01-04  ###
        ###BUILD INDEX###
        ###MAP###
Traceback (most recent call last):
  File "/opt/LoReAn/code/lorean.py", line 552, in <module>
    main()
  File "/opt/LoReAn/code/lorean.py", line 180, in main
    args.verbose, Fflag=True)
  File "/opt/LoReAn/code/mapping.py", line 398, in gmap
    wd, Fflag, type_out, verbose)
  File "/opt/LoReAn/code/mapping.py", line 130, in gmap_map
    filename = longest_cds(list_fasta[0][1], list_fasta[1][1], verbose, working_dir, filename)
  File "/opt/LoReAn/code/mapping.py", line 188, in longest_cds
    db = gffutils.create_db(gff_file, ':memory:', merge_strategy='create_unique', keep_order=True, transform=transform)
  File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 1288, in create_db
    c.create()
  File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 504, in create
    self._populate_from_lines(self.iterator)
  File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 625, in _populate_from_lines
    raise ValueError("No lines parsed -- was an empty file provided?")
ValueError: No lines parsed -- was an empty file provided?

It seems to stop when aligning transcripts using GMAP. I then tried appending -mm to use minimap.

$ lorean.py -a -rp repeats.scaffold3.bed -sr scaffold3.short_1.fastq,scaffold3.short_2.fastq  -lr scaffold3.long.fasta  -pr scaffold3.prot.fasta -sp crispa scaffold3.fasta -d -f -mg -t 60 --keep_tmp -mm

 ### LOREAN IS CHECKING THAT GMAP IS CORRECTLY BUILD ###

###GMAP MAPPING  STARTED AT:    09:15:54 01-04  ###
        ###BUILD INDEX###
GMAP database existed already: /mnt/nas1/hhl/phellinus/cDNA/lorean/LoReAn_Example/Crispa/LoReAn_annotation/run//gmap_output/chr8.testGMAP.fasta_GMAPindex --- skipping  ###MAP###
GMAP done already: /mnt/nas1/hhl/phellinus/cDNA/lorean/LoReAn_Example/Crispa/LoReAn_annotation/run//gmap_output/gmap.long_reads.test.sam --- skipping
 ### GMAP IS CORRECTLY BUILD ###

###MASKING THE GENOME STARTED AT:       09:15:53 01-04  ###

###STAR MAPPING  STARTED AT:    09:15:54 01-04  ###
        ###BUILD INDEX###
STAR index existed already: /mnt/nas1/hhl/phellinus/cDNA/lorean/LoReAn_Example/Crispa/LoReAn_annotation/run//STAR/scaffold3.fasta.masked.fasta.rename.fasta_STARindex/SAindex --- skipping      ###MAP###
STAR alignment file existed already: /mnt/nas1/hhl/phellinus/cDNA/lorean/LoReAn_Example/Crispa/LoReAn_annotation/run//STAR/STAR_shortreadsAligned.out.bam --- skipping
        ###SAM to BAM###
BAM file existed already: /mnt/nas1/hhl/phellinus/cDNA/lorean/LoReAn_Example/Crispa/LoReAn_annotation/run//STAR/short_reads_mapped.bam --- skipping
        ###SORTING BAM###

###TRINITY STARTS AT:   09:17:42 01-04  ###
Trinity-GG file existed already: /mnt/nas1/hhl/phellinus/cDNA/lorean/LoReAn_Example/Crispa/LoReAn_annotation/run//Trinity/trinity_out_dir/Trinity-GG.fasta --- skipping

###GMAP MAPPING  STARTED AT:    09:17:42 01-04  ###
        ###BUILD INDEX###
GMAP database existed already: /mnt/nas1/hhl/phellinus/cDNA/lorean/LoReAn_Example/Crispa/LoReAn_annotation/run//gmap_output/scaffold3.fasta.masked.fasta.rename.fasta_GMAPindex --- skipping    ###MAP###
Traceback (most recent call last):
  File "/opt/LoReAn/code/lorean.py", line 552, in <module>
    main()
  File "/opt/LoReAn/code/lorean.py", line 180, in main
    args.verbose, Fflag=True)
  File "/opt/LoReAn/code/mapping.py", line 398, in gmap
    wd, Fflag, type_out, verbose)
  File "/opt/LoReAn/code/mapping.py", line 130, in gmap_map
    filename = longest_cds(list_fasta[0][1], list_fasta[1][1], verbose, working_dir, filename)
  File "/opt/LoReAn/code/mapping.py", line 188, in longest_cds
    db = gffutils.create_db(gff_file, ':memory:', merge_strategy='create_unique', keep_order=True, transform=transform)
  File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 1288, in create_db
    c.create()
  File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 504, in create
    self._populate_from_lines(self.iterator)
  File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 625, in _populate_from_lines
    raise ValueError("No lines parsed -- was an empty file provided?")
ValueError: No lines parsed -- was an empty file provided?

It did not swap to minimap and encounter the same error?

cgjosephlee commented 5 years ago 
$ tail gmap_output/gmap_map.log
Checking compiler options for SSE4.2: 6B8B4567 __builtin_clz=1 __builtin_ctz=0 _mm_popcnt_u32=17 __builtin_popcount=17
Finished checking compiler assumptions
Looking for index files in directory /mnt/nas1/hhl/test/cDNA/lorean/LoReAn_Example/Crispa/LoReAn_annotation/run//gmap_output//scaffold3.fasta.masked.fasta.rename.fasta_GMAPindex
  Pointers file is scaffold3.fasta.masked.fasta.rename.fasta_GMAPindex.ref153offsets64meta
  Offsets file is scaffold3.fasta.masked.fasta.rename.fasta_GMAPindex.ref153offsets64strm
  Positions file is scaffold3.fasta.masked.fasta.rename.fasta_GMAPindex.ref153positions
Offsets compression type: bitpack64
Allocating memory for ref offset pointers, kmer 15, interval 3...done (134,217,744 bytes)
Starting alignment
Processed 0 queries in 0.00 seconds (-nan queries/sec)

No queries were passed to GMAP and the file size of Trinity/trinity_out_dir/Trinity-GG.fasta is 0. Seems to fail at trinity stage?

lfaino commented 5 years ago

Working on it. I will write ASAP i will find the problem

lfaino commented 5 years ago

Dear @cgjosephlee and @ncgrjsmiao , i did build a new image with some change. It looks that i fix the problems. you can try by running:

docker run -it --rm -v $PWD:/data lfaino/lorean:noiIPRS createUser.py $USER $UID

please let me know if it works that i came move this as default image Cheers Luigi

cgjosephlee commented 5 years ago

Hi @lfaino , Thank you for the reply. Unfortunately it stopped at the same stage with same error message, and Trinity-GG.fasta is still zero-size. Any additional info I can provide?

Best Joseph

lfaino commented 5 years ago

We need to check if mysql started.

you can test this by running: docker run -it --rm -v $PWD:/data lfaino/lorean:noiIPRS bash once the image start, you can run:

mysql -u root -p and when ask the password you can type lorean

you should be in mysql. if you get an error, let me know Cheers Luigi

cgjosephlee commented 5 years ago

Gocha!

root@332f11a6b0ee:/data# mysql -u root -p
Enter password:
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
lfaino commented 5 years ago

do you have installed mysql on your system?

cgjosephlee commented 5 years ago

Yes, but it seems that I cannot access it since I have no root privilege.

$ mysql --help
mysql  Ver 14.14 Distrib 5.7.25, for Linux (x86_64) using  EditLine wrapper
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
lfaino commented 5 years ago

Hi @cgjosephlee , can you test again. I removed mysql and now it works with my data.

please let me know.

Cheers Luigi

zcrself commented 5 years ago

Dear @cgjosephlee and @ncgrjsmiao , i did build a new image with some change. It looks that i fix the problems. you can try by running:

docker run -it --rm -v $PWD:/data lfaino/lorean:noiIPRS createUser.py $USER $UID

please let me know if it works that i came move this as default image Cheers Luigi

_With this docker://lfaino/lorean:noiIPRS, running Singularity 3.1.0 Using LoReAnExample, completely finished! But as for my test data, there are some errors as follows, Using docker://lfaino/lorean, there are no errors like below, but error is "ValueError: No lines parsed -- was an empty file provided?"

`cp: cannot stat '/db2/ncgr/maozhu/Partial_Test/LoReAn_annotation/run/PASA/annotation.pasa_assemblies.gff3': No such file or directory Traceback (most recent call last): File "/opt/LoReAn/code/prepareEvmInputs.py", line 105, in move_single_file subprocess.check_call(args) File "/usr/lib/python3.5/subprocess.py", line 581, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['cp', '/db2/ncgr/maozhu/Partial_Test/LoReAn_annotation/run/PASA/annotation.pasa_assemblies.gff3', '/db2/ncgr/maozhu/Partial_Test/LoReAn_annotation/run//evm_inputs/']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/LoReAn/code/lorean.py", line 551, in main() File "/opt/LoReAn/code/lorean.py", line 363, in main list_soft, pred_file, transcript_file, protein_file = inputEvm.group_EVM_inputs(evm_inputs_dir, evm_inputs) File "/opt/LoReAn/code/prepareEvmInputs.py", line 234, in group_EVM_inputs move_EVM_inputs(evm_dir, inputs) File "/opt/LoReAn/code/prepareEvmInputs.py", line 161, in move_EVM_inputs new_files = move_single_file(filename, key, evm_dir, new_files) File "/opt/LoReAn/code/prepareEvmInputs.py", line 110, in move_single_file raise NameError('') NameError`

zcrself commented 5 years ago

Exit at "MOVING IMPORTANT FILES"

PASA STARTS AT: 08:09:28 06-04

AUGUSTUS, GENEMARK-ES AND AAT STARTED AT:08:47:55 06-04

    ###RUNNING AUGUSTUS ###
    ###RUNNING AAT ###
    ###RUNNING GENEMARK ###
    ###CONVERTING AUGUSTUS TO GFF3###
    ###CONVERTING GENEMARK TO GFF3###

EVM STARTED AT: 14:29:52 06-04

    ###MOVING IMPORTANT FILES###
lfaino commented 5 years ago

Hi @cgjosephlee , in the past few days I made few changes and few (about 50) new images. I see that that the image that you are using is a bit old. Please, remove the LoReAn _annotation folder and the lorean image, dowload again the image and run again. let me know how it goes.

Cheers Luigi

zcrself commented 5 years ago

ls -l /db2/ncgr/maozhu/Partial_Test/LoReAn_annotation/run/PASA/:

drwxr-xr-x 8 ncgr ncgr 4096 Apr 6 16:47 ./ drwxr-xr-x 14 ncgr ncgr 4096 Apr 6 22:29 ../ -rw-r--r-- 1 ncgr ncgr 239 Apr 6 16:09 alignAssembly.config -rw-r--r-- 1 ncgr ncgr 6240457 Apr 6 16:25 alignment.validations.output -rw-r--r-- 1 ncgr ncgr 5098995 Apr 6 16:33 annotation.sqlite.assemblies.fasta -rw-r--r-- 1 ncgr ncgr 4749973 Apr 6 16:25 annotation.sqlite.failed_gmap_alignments.gff3 -rw-r--r-- 1 ncgr ncgr 9816379 Apr 6 16:25 annotation.sqlite.failed_gmap_alignments.gtf -rw-r--r-- 1 ncgr ncgr 6068647 Apr 6 16:26 annotation.sqlite.pasa_alignment_assembly_building.ascii_illustrations.out -rw-r--r-- 1 ncgr ncgr 399257 Apr 6 16:47 annotation.sqlite.pasa_assemblies.bed -rw-r--r-- 1 ncgr ncgr 903605 Apr 6 16:47 annotation.sqlite.pasa_assemblies_described.txt -rw-r--r-- 1 ncgr ncgr 1414860 Apr 6 16:47 annotation.sqlite.pasa_assemblies.gff3 -rw-r--r-- 1 ncgr ncgr 2111630 Apr 6 16:47 annotation.sqlite.pasa_assemblies.gtf -rw-r--r-- 1 ncgr ncgr 1277577 Apr 6 16:25 annotation.sqlite.valid_gmap_alignments.bed -rw-r--r-- 1 ncgr ncgr 3255262 Apr 6 16:25 annotation.sqlite.valid_gmap_alignments.gff3 -rw-r--r-- 1 ncgr ncgr 6730264 Apr 6 16:25 annotation.sqlite.valid_gmap_alignments.gtf -rw-r--r-- 1 ncgr ncgr 13780996 Apr 6 16:14 gmap.spliced_alignments.gff3 -rw-r--r-- 1 ncgr ncgr 0 Apr 6 16:14 gmap.spliced_alignments.gff3.completed drwxr-xr-x 2 ncgr ncgr 4096 Apr 6 16:47 pasa_annotation.sqlite_SQLite_chkpts/ -rw-r--r-- 1 ncgr ncgr 6100 Apr 6 16:47 pasa_annotation.sqlite_SQLite_chkpts.cmds_log -rw-r--r-- 1 ncgr ncgr 4906250 Apr 6 16:47 pasa.err.log -rw-r--r-- 1 ncgr ncgr 414 Apr 6 16:47 pasa.out.log drwxr-xr-x 2 ncgr ncgr 4096 Apr 6 16:26 pasa_run.log.dir/ -rw-r--r-- 1 ncgr ncgr 303 Apr 6 16:14 pipeliner.6340.cmds -rw-r--r-- 1 ncgr ncgr 3321 Apr 6 16:20 pipeliner.6548.cmds -rw-r--r-- 1 ncgr ncgr 3138 Apr 6 16:20 pipeliner.7036.cmds drwxr-xr-x 3 ncgr ncgr 4096 Apr 6 16:11 test_scaffold.fa.masked.fasta.rename.fasta.gmap/ -rw-r--r-- 1 ncgr ncgr 1988655 Apr 6 16:20 Trinity-GG.fasta.transdecoder.bed -rw-r--r-- 1 ncgr ncgr 12196018 Apr 6 16:21 Trinity-GG.fasta.transdecoder.cds drwxr-xr-x 3 ncgr ncgr 4096 Apr 6 16:20 Trinity-GG.fasta.transdecoder_dir/ drwxr-xr-x 2 ncgr ncgr 4096 Apr 6 16:21 Trinity-GG.fasta.transdecoder_dir.checkpoints/ drwxr-xr-x 2 ncgr ncgr 4096 Apr 6 16:16 Trinity-GG.fasta.transdecoder_dir.checkpoints_longorfs/ -rw-r--r-- 1 ncgr ncgr 8775031 Apr 6 16:20 Trinity-GG.fasta.transdecoder.gff3 -rw-r--r-- 1 ncgr ncgr 76611 Apr 6 16:21 Trinity-GG.fasta.transdecoder.gff3.fl_accs -rw-r--r-- 1 ncgr ncgr 5253718 Apr 6 16:20 Trinity-GG.fasta.transdecoder.pep

lfaino commented 5 years ago

is this the output after that you re-downloaded the image?

zcrself commented 5 years ago

is this the output after that you re-downloaded the image? I have download docker://lfaino/lorean some days ago, Because you update your docker image, then I download this lfaino/lorean:noiIPRS image firstly ( I do not delete lfaino/lorean because of different image)

zcrself commented 5 years ago

This is the output after I download this image docker://lfaino/lorean:noiIPRS firstly

Could you tell me that the differences between docker://lfaino/lorean and docker://lfaino/lorean:noiIPRS?

On Sun, Apr 7, 2019 at 3:47 AM Luigi Faino notifications@github.com wrote:

is this the output after that you re-downloaded the image?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-480532424, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuEkiLhTILYIRmhq4iIrIP8setSIsks5vePm9gaJpZM4cLtZj .

lfaino commented 5 years ago

I saw an old error (on my side) in your images: the image is not anymore docker://lfaino/lorean:noiIPRS but docker://lfaino/lorean:noIPRS. if you notice, there is not anymore the i before IPRS. it was a mistake from my side. The latest image is called docker://lfaino/lorean:noIPRS. There are few differences between docker://lfaino/lorean and docker://lfaino/lorean:noIPRS: 1) docker://lfaino/lorean:noIPRS does not require mysql anymore. this was a pain before because LoReAn need to start a service in the image and it is a bit difficult on some host systems. now PASA uses sqlite and it is managed better by the image 2) I update GeneMark. now i have the latest version in docker://lfaino/lorean:noIPRS 3) gffread tool was updated and few change need to be made to use the new version in docker://lfaino/lorean:noIPRS

these are the main changes.

Sorry for the problem with the naming. Please dowload the new version using docker run -it --rm -v $PWD:/data lfaino/lorean:noIPRS createUser.py $USER $UID and test again

zcrself commented 5 years ago

Thank for your good job and quick reply. Now I re-download docker://lfaino/lorean:noIPRS, then re-run my job. Tomorrow I will tell you my result of running lorean:noIPRS!

Best regards!

On Sun, Apr 7, 2019 at 3:04 PM Luigi Faino notifications@github.com wrote:

I saw an old error (on my side) in your images: the image is not anymore docker://lfaino/lorean:noiIPRS but docker://lfaino/lorean:noIPRS. if you notice, there is not anymore the i before IPRS. it was a mistake from my side. The latest image is called docker://lfaino/lorean:noIPRS. There are few differences between docker://lfaino/lorean and docker://lfaino/lorean:noIPRS:

  1. docker://lfaino/lorean:noIPRS does not require mysql anymore. this was a pain before because LoReAn need to start a service in the image and it is a bit difficult on some host systems. now PASA uses sqlite and it is managed better by the image
  2. I update GeneMark. now i have the latest version in docker://lfaino/lorean:noIPRS
  3. gffread tool was updated and few change need to be made to use the new version in docker://lfaino/lorean:noIPRS

these are the main changes.

Sorry for the problem with the naming. Please dowload the new version using docker run -it --rm -v $PWD:/data lfaino/lorean:noIPRS createUser.py $USER $UID and test again

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-480564987, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuAphjSMpD5D2DZ_zA8xW2VwbQzs-ks5veZiRgaJpZM4cLtZj .

zcrself commented 5 years ago

Using the image docker://lfaino/lorean:noIPRS

Errors as follows:

RUNNING iASSEMBLER 02:07:27 08-04

Traceback (most recent call last): File "/opt/LoReAn/code//lorean.py", line 563, in main() File "/opt/LoReAn/code//lorean.py", line 440, in main adapter_value, threads_use, args.adapter_match_score, ref_rename, UnboundLocalError: local variable 'adapter_value' referenced before assignment

How to fix it ? Thanks a lot!

On Sun, Apr 7, 2019 at 10:56 PM Smith Waterman zcrself@gmail.com wrote:

Thank for your good job and quick reply. Now I re-download docker://lfaino/lorean:noIPRS, then re-run my job. Tomorrow I will tell you my result of running lorean:noIPRS!

Best regards!

On Sun, Apr 7, 2019 at 3:04 PM Luigi Faino notifications@github.com wrote:

I saw an old error (on my side) in your images: the image is not anymore docker://lfaino/lorean:noiIPRS but docker://lfaino/lorean:noIPRS. if you notice, there is not anymore the i before IPRS. it was a mistake from my side. The latest image is called docker://lfaino/lorean:noIPRS. There are few differences between docker://lfaino/lorean and docker://lfaino/lorean:noIPRS:

  1. docker://lfaino/lorean:noIPRS does not require mysql anymore. this was a pain before because LoReAn need to start a service in the image and it is a bit difficult on some host systems. now PASA uses sqlite and it is managed better by the image
  2. I update GeneMark. now i have the latest version in docker://lfaino/lorean:noIPRS
  3. gffread tool was updated and few change need to be made to use the new version in docker://lfaino/lorean:noIPRS

these are the main changes.

Sorry for the problem with the naming. Please dowload the new version using docker run -it --rm -v $PWD:/data lfaino/lorean:noIPRS createUser.py $USER $UID and test again

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-480564987, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuAphjSMpD5D2DZ_zA8xW2VwbQzs-ks5veZiRgaJpZM4cLtZj .

zcrself commented 5 years ago

My running command list is : " nohup lorean.py -a -rp repeats.fix.bed -sr clean_R_1.fastq,clean_R_2.fastq -lr Long.fasta -pr rice_bamboo_prot.fasta -sp bamboo test_scaffold.fa -d -f -mg -t 4 --keep_tmp 1>1.out 2>2.out &"

On Mon, Apr 8, 2019 at 2:36 PM Smith Waterman zcrself@gmail.com wrote:

Using the image docker://lfaino/lorean:noIPRS

Errors as follows:

RUNNING iASSEMBLER 02:07:27 08-04

Traceback (most recent call last): File "/opt/LoReAn/code//lorean.py", line 563, in main() File "/opt/LoReAn/code//lorean.py", line 440, in main adapter_value, threads_use, args.adapter_match_score, ref_rename, UnboundLocalError: local variable 'adapter_value' referenced before assignment

How to fix it ? Thanks a lot!

On Sun, Apr 7, 2019 at 10:56 PM Smith Waterman zcrself@gmail.com wrote:

Thank for your good job and quick reply. Now I re-download docker://lfaino/lorean:noIPRS, then re-run my job. Tomorrow I will tell you my result of running lorean:noIPRS!

Best regards!

On Sun, Apr 7, 2019 at 3:04 PM Luigi Faino notifications@github.com wrote:

I saw an old error (on my side) in your images: the image is not anymore docker://lfaino/lorean:noiIPRS but docker://lfaino/lorean:noIPRS. if you notice, there is not anymore the i before IPRS. it was a mistake from my side. The latest image is called docker://lfaino/lorean:noIPRS. There are few differences between docker://lfaino/lorean and docker://lfaino/lorean:noIPRS:

  1. docker://lfaino/lorean:noIPRS does not require mysql anymore. this was a pain before because LoReAn need to start a service in the image and it is a bit difficult on some host systems. now PASA uses sqlite and it is managed better by the image
  2. I update GeneMark. now i have the latest version in docker://lfaino/lorean:noIPRS
  3. gffread tool was updated and few change need to be made to use the new version in docker://lfaino/lorean:noIPRS

these are the main changes.

Sorry for the problem with the naming. Please dowload the new version using docker run -it --rm -v $PWD:/data lfaino/lorean:noIPRS createUser.py $USER $UID and test again

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-480564987, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuAphjSMpD5D2DZ_zA8xW2VwbQzs-ks5veZiRgaJpZM4cLtZj .

lfaino commented 5 years ago

The easiest way is to provide a file with the adapter sequence. Are you using PacBio or Nanopore data?

cgjosephlee commented 5 years ago

Still no luck after re-download latest image. I doubt it was stuck at trinity stage. trinity.log

--------------------------------------------------------------------------------
------------ Trinity Phase 2: Assembling Clusters of Reads ---------------------
--------------------------------------------------------------------------------

Monday, April 8, 2019: 02:03:26 CMD: /opt/LoReAn/third_party/software/Trinity/trinity-plugins/BIN/ParaFly -c trinity_GG.cmds -CPU 20 -v -shuffle
Number of Commands: 581
^Msucceeded(1)   0.172117% completed.    ^Msucceeded(2)   0.344234% completed.    ^Msucceeded(3)   0.516351% completed.    ^Msucceeded(4)   0.688468% completed.    ^Msucceeded(5)   0.860585% completed.    ^Msucceeded(6)   1.0327% completed.    ^Msucceeded(7)   1.20482% completed.    ^Msucceeded(8)   1.37694% complete

All commands completed successfully. :-)

Monday, April 8, 2019: 02:07:38 CMD: find Dir_*  -name '*inity.fasta'  | /opt/LoReAn/third_party/software/Trinity/util/support_scripts/GG_partitioned_trinity_aggregator.pl TRINITY_GG > /data/LoReAn_Example/Crispa/LoReAn_annotation/run/Trinity/trinity_out_dir/Trinity-GG.fasta.tmp
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/0/y.reads.out.Trinity.fasta': No such file or directory
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/0/reads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/0/y.reads.out.Trinity.fasta is indicated to not exist.
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/0/eads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/0/reads.out.Trinity.fasta is indicated to not exist.
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/0/.reads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/0/eads.out.Trinity.fasta is indicated to not exist.
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/0/ty.reads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/0/.reads.out.Trinity.fasta is indicated to not exist.
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/1/ty.reads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/0/ty.reads.out.Trinity.fasta is indicated to not exist.
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/2/ity.reads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/1/ty.reads.out.Trinity.fasta is indicated to not exist.
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/2/nity.reads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/2/ity.reads.out.Trinity.fasta is indicated to not exist.
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/2/ty.reads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/2/nity.reads.out.Trinity.fasta is indicated to not exist.
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/3/nity.reads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/2/ty.reads.out.Trinity.fasta is indicated to not exist.
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/4/nity.reads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/3/nity.reads.out.Trinity.fasta is indicated to not exist.
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/5/nity.reads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/4/nity.reads.out.Trinity.fasta is indicated to not exist.
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/6/nity.reads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/5/nity.reads.out.Trinity.fasta is indicated to not exist.
find: 'Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/7/nity.reads.out.Trinity.fasta': No such file or directory
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/6/nity.reads.out.Trinity.fasta is indicated to not exist.
ERROR, filename: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/7/nity.reads.out.Trinity.fasta is indicated to not exist.
Monday, April 8, 2019: 02:07:38 CMD: /opt/LoReAn/third_party/software/Trinity/util/support_scripts/get_Trinity_gene_to_trans_map.pl /data/LoReAn_Example/Crispa/LoReAn_annotation/run/Trinity/trinity_out_dir/Trinity-GG.fasta > /data/LoReAn_Example/Crispa/LoReAn_annotation/run/Trinity/trinity_out_dir/Trinity-GG.fasta.ge

Finished. See /data/LoReAn_Example/Crispa/LoReAn_annotation/run/Trinity/trinity_out_dir/Trinity-GG.fasta for reconstructed transcripts
lfaino commented 5 years ago

do you mind to share your test data? I will test and see if i can find the error and fix it

cgjosephlee commented 5 years ago

The test data is LoReAn_Example/Crispa downloaded at https://github.com/lfaino/LoReAn_Example.

lfaino commented 5 years ago

@cgjosephlee ok. do you have /data/LoReAn_Example/Crispa/LoReAn_annotation/run/Trinity/trinity_out_dir/Trinity-GG.fasta file? is it empty or full? can you tell me about the system that you are using (amount of Memory and CPU)?

lfaino commented 5 years ago

@zcrself; do you mind to share your test data? I will test and see if i can find the error and fix it

lfaino commented 5 years ago

@cgjosephlee: Dir_STAR_shortreadsAligned.out.bam.sorted.bam.minC1.gff/seq1/4/nity.reads.out.Trinity.fasta; this is strange... the file is truncated (4/nity.reads.out.Trinity.fasta).... no idea why

cgjosephlee commented 5 years ago

My system is Ubuntu 16.04 with 64 cpus and 128gb ram. I specified 40 cpus for the test.

docker run -it --rm -v $PWD:/data lfaino/lorean:noIPRS createUser.py $USER $UID
lorean.py -a -rp repeats.scaffold3.bed -sr scaffold3.short_1.fastq,scaffold3.short_2.fastq  -lr scaffold3.long.fasta  -pr scaffold3.prot.fasta -sp crispa scaffold3.fasta -d -f -mg -t 40 --keep_tmp

I previously tried on our NAS system and had no luck. I then tried on a local file system and it running now. Trinity-GG.fasta is full this time. Probably a conflict between docker and file system or symbolic links?

zcrself commented 5 years ago

There are no adapter sequences in my long sequence(PacBio data),

Such as: @transcript/53243 full_length_coverage=2;length=662;numsubreads=19 ACACCGCGAAACCAAACACATTTTGCCATCCATCTCTCGCCTCCGGCTCCCCCAAGTTCCCAATCTCGCGACTCCATCCACGCCCGATCGAATCCGTGATGGCGCCCAAGGCCGAGAAGAAGCCGCGGAGAAGAAGCCCGTGGAGAAGAAGGAGAAGGCGGAGAAGCGCGTCCCCGGCGCCAAGGAGGGCGGCGCGGACAAGAAGGGGAAGAAGAAGGCCAAGAAGAGCGTGGAGACGTACAAGATCTACATCTTCAAGGTGCTCAAGCAGGTGCACCCGGACATCGGCATCTCCTCCAAGGCATGTCTATCATGAACTCCTTCATCAACGACATCTTCGAGAAGCTCGCCCAGGAGGCCGCCCGCCTCGCCCGCTATAACAAGAAGCCAACCATCACCTCCCGCGAGATCCAGACCTCAGTGCGCCTCGTCCTCCCCGGCGAGCTCGCCAAGCACGCCGTCTCGGAGGGCACCAAGGCCGTCACCAAGTTCACGTCCTCTTAGGCATGCAGAGCCTTGGCGGTTTACCATTTGGTTTGGTTCGGTTTGGTCCCATGTTTAGAGCATAGCGTTAGATTTCAAATATGGAACGATGTAGAATGAGTTGAATTTGCTGAATGAAAGAATGATCTCGAGTTACAAAAAAAAAAAAAAAAAAAAAA + bEO:cRLS9l[@X&vsOX=ZGxyjfI~eNR~usc~]gj}MU~f~h~,t|~LuliZ{\vur~[~~vt~Q~x\~~p~O~)G~Z~8~~Pv;~yIm~N_)~~a~w~iz~y~ou~N~A~&p8>;~|~~bwe~n~{e|[i~L~t~x~z~.e~a~~]xb~~ic~|~f~2w|zZ~yz~?>~$~~z~z~x~~e|~~h~~p~v~>~~kU~t~~7~~p~Z~}~O~T;~~~va~~{O~~|f,~~~}~\~t~~s~b~~lu~~u~|~k~:~_~{~K~~~>~~v~r~kX~~N~y~~~~{tb~a~U~~~~l~~W~p~~~~w-~~~~~~~

On Mon, Apr 8, 2019 at 2:40 PM Luigi Faino notifications@github.com wrote:

The easiest way is to provide a file with the adapter sequence. Are you using PacBio or Nanopore data?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-480703686, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuMhVrBHD2KzePZ3RKncIcHvlh5grks5veuRzgaJpZM4cLtZj .

lfaino commented 5 years ago

@zcrself do you know why? did you remove them? Anyway, can you just add a poly T? like 15T in a fasta file and like:

polyT TTTTTTTTTTTTTTT

and pass this file with -a option? let me know Luigi

zcrself commented 5 years ago

My new running command line is "lorean.py -rp repeats.fix.bed -sr test_sr_1.fastq,test_sr_2.fastq -lr Long.fasta -pr rice_bamboo_prot.fasta -sp rice test_scaffold.fa -t 4 --keep_tmp"

Because of no adapter seq, delete '-a' Because of having masked test_scaffold.fa, delete '-mg' Because of bamboo data, delete '-f' Because of 'no stranded' mode, delete '-d'

Are there any questions about my new command line ?

On Mon, Apr 8, 2019 at 3:58 PM Smith Waterman zcrself@gmail.com wrote:

There are no adapter sequences in my long sequence(PacBio data),

Such as: @transcript/53243 full_length_coverage=2;length=662;num_subreads=19

ACACCGCGAAACCAAACACATTTTGCCATCCATCTCTCGCCTCCGGCTCCCCCAAGTTCCCAATCTCGCGACTCCATCCACGCCCGATCGAATCCGTGATGGCGCCCAAGGCCGAGAAGAAGCCGCGGAGAAGAAGCCCGTGGAGAAGAAGGAGAAGGCGGAGAAGCGCGTCCCCGGCGCCAAGGAGGGCGGCGCGGACAAGAAGGGGAAGAAGAAGGCCAAGAAGAGCGTGGAGACGTACAAGATCTACATCTTCAAGGTGCTCAAGCAGGTGCACCCGGACATCGGCATCTCCTCCAAGGCATGTCTATCATGAACTCCTTCATCAACGACATCTTCGAGAAGCTCGCCCAGGAGGCCGCCCGCCTCGCCCGCTATAACAAGAAGCCAACCATCACCTCCCGCGAGATCCAGACCTCAGTGCGCCTCGTCCTCCCCGGCGAGCTCGCCAAGCACGCCGTCTCGGAGGGCACCAAGGCCGTCACCAAGTTCACGTCCTCTTAGGCATGCAGAGCCTTGGCGGTTTACCATTTGGTTTGGTTCGGTTTGGTCCCATGTTTAGAGCATAGCGTTAGATTTCAAATATGGAACGATGTAGAATGAGTTGAATTTGCTGAATGAAAGAATGATCTCGAGTTACAAAAAAAAAAAAAAAAAAAAAA +

bEO:cRLS9l[@X&vsOX=ZGxyjfI~eNR~usc~]gj}MU~f~h~,t|~LuliZ{\vur~[v_t~Q~x~~\~~p~O~)G~Z~8~~Pv;~yIm~N_)~~a~w~iz~y~ou~N~A~&p8>;~|~~bwe~n~{e|[i~L~t~x~z~.e~a~~]xb~~ic~|~f~2w|zZ~yz~?>~$~~z~z~x~~e|~~h~~p~v~>~~kU~t~~7~~p~Z~}~O~T;~~~va~~{O~~|f,~~~}~\~t~~s~b~~lu~~u~|~k~:~_~{~K~~~>~~v~r~kX~~N~y~~~~{tb~a~U~~~~l~~W~p~~~~w-~~~~~~~

On Mon, Apr 8, 2019 at 2:40 PM Luigi Faino notifications@github.com wrote:

The easiest way is to provide a file with the adapter sequence. Are you using PacBio or Nanopore data?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-480703686, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuMhVrBHD2KzePZ3RKncIcHvlh5grks5veuRzgaJpZM4cLtZj .

lfaino commented 5 years ago

@zcrself: it looks good. however, I never tested with "nohup". I would suggest to use screen, instead. Additionally, can you please use: lorean -rp repeats.fix.bed -sr test_sr_1.fastq,test_sr_2.fastq -lr Long.fasta -pr rice_bamboo_prot.fasta -sp rice test_scaffold.fa -t 4 --keep_tmp

the command changes in "lorean" rather than "lorean.py"

Additionally, can you remove and re-downlaod the image "/lfaino/lorean:noIPRS"? I made some change based on your previous errors in a new image.

thanks for all the help and patience Cheers Luigi

zcrself commented 5 years ago

After I see at the result of this running, if failed, I will remove and re-downlaod the image "/lfaino/lorean:noIPRS", then run command line that "lorean -rp repeats.fix.bed -sr test_sr_1.fastq,test_sr_2.fastq -lr Long.fasta -pr rice_bamboo_prot.fasta -sp rice test_scaffold.fa -t 4 --keep_tmp"

Tomorrow I will tell you the result of running

Best regards!

On Mon, Apr 8, 2019 at 4:31 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself: it looks good. however, I never tested with "nohup". I would suggest to use screen, instead. Additionally, can you please use: lorean -rp repeats.fix.bed -sr test_sr_1.fastq,test_sr_2.fastq -lr Long.fasta -pr rice_bamboo_prot.fasta -sp rice test_scaffold.fa -t 4 --keep_tmp

the command changes in "lorean" rather than "lorean.py"

Additionally, can you remove and re-downlaod the image "/lfaino/lorean:noIPRS"? I made some change based on your previous errors in a new image.

thanks for all the help and patience Cheers Luigi

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-480736920, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuL_z63nyQFgGhm7dFDac6L6iVAy6ks5vev5dgaJpZM4cLtZj .

zcrself commented 5 years ago

Still errors below like before. I will remove and re-downlaod the image "/lfaino/lorean:noIPRS", then re-run the command line that " lorean -rp repeats.fix.bed -sr test_sr_1.fastq,test_sr_2.fastq -lr Long.fasta -pr rice_bamboo_prot.fasta -sp rice test_scaffold.fa -t 4 --keep_tmp"

Traceback (most recent call last): File "/opt/LoReAn/code//lorean.py", line 563, in main() File "/opt/LoReAn/code//lorean.py", line 440, in main adapter_value, threads_use, args.adapter_match_score, ref_rename, UnboundLocalError: local variable 'adapter_value' referenced before assignment

On Mon, Apr 8, 2019 at 4:31 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself: it looks good. however, I never tested with "nohup". I would suggest to use screen, instead. Additionally, can you please use: lorean -rp repeats.fix.bed -sr test_sr_1.fastq,test_sr_2.fastq -lr Long.fasta -pr rice_bamboo_prot.fasta -sp rice test_scaffold.fa -t 4 --keep_tmp

the command changes in "lorean" rather than "lorean.py"

Additionally, can you remove and re-downlaod the image "/lfaino/lorean:noIPRS"? I made some change based on your previous errors in a new image.

thanks for all the help and patience Cheers Luigi

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-480736920, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuL_z63nyQFgGhm7dFDac6L6iVAy6ks5vev5dgaJpZM4cLtZj .

lfaino commented 5 years ago

@zcrself I as able to reproduce the error. I will fix it, build a new image and write to you

zcrself commented 5 years ago

I do be grateful to you for doing this!

Thank you very much!

On Tue, Apr 9, 2019 at 2:12 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself I as able to reproduce the error. I will fix it, build a new image and write to you

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481116351, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuL9QCoNKeyMpCmuN_q2J6D-ELomPks5vfC9cgaJpZM4cLtZj .

zcrself commented 5 years ago

Running command lines as you tell me, but still error like before

  1. remove and re-downlaod the image "/lfaino/lorean:noIPRS"
  2. no "nohup", use screen
  3. "lorean" rather than "lorean.py"

EVM STARTED AT: 09:33:25 09-04

MOVING IMPORTANT FILES

CONCATENATING FILES

PARTITIONING THE INPUTS

GROUPING COMMANDS

RUNNING EVM

COMBINING PARTITIONS

CONVERTING TO GFF3

UPDATE WITH PASA DATABASE STARTED AT: 09:41:49 09-04

CREATING CONFIGURATION FILE

2019-04-09T09:41:49.379584Z 3 [Note] Access denied for user 'xxxxxx'@'localhost' (using password: YES)

LOADING GFF3 FILE INTO DATABASE

UPDATING GFF3 FILE

PARSING OUTPUT

EVM GFF3 STATS

parsed genome node DAGs: 1897 sequence regions: 20 (total length: 48719482) genes: 1876 protein-coding genes: 1876 mRNAs: 1979 protein-coding mRNAs: 1979 exons: 8866 CDSs: 8696 introns: 20661

RUNNING iASSEMBLER 10:02:50 09-04

Traceback (most recent call last): File "/opt/LoReAn/code/lorean.py", line 563, in main() File "/opt/LoReAn/code/lorean.py", line 440, in main adapter_value, threads_use, args.adapter_match_score, ref_rename, UnboundLocalError: local variable 'adapter_value' referenced before assignment

On Mon, Apr 8, 2019 at 4:31 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself: it looks good. however, I never tested with "nohup". I would suggest to use screen, instead. Additionally, can you please use: lorean -rp repeats.fix.bed -sr test_sr_1.fastq,test_sr_2.fastq -lr Long.fasta -pr rice_bamboo_prot.fasta -sp rice test_scaffold.fa -t 4 --keep_tmp

the command changes in "lorean" rather than "lorean.py"

Additionally, can you remove and re-downlaod the image "/lfaino/lorean:noIPRS"? I made some change based on your previous errors in a new image.

thanks for all the help and patience Cheers Luigi

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-480736920, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuL_z63nyQFgGhm7dFDac6L6iVAy6ks5vev5dgaJpZM4cLtZj .

lfaino commented 5 years ago

@zcrself The images is building and it will be ready in 1 hr. I will write to you Cheers Luigi

zcrself commented 5 years ago

great! Thank for your doing!

On Tue, Apr 9, 2019 at 8:41 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself The images is building and it will be ready in 1 hr. I will write to you Cheers Luigi

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481234541, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuC2W7hJkMNQwLfAAq_OW96m4QxGfks5vfIqKgaJpZM4cLtZj .

lfaino commented 5 years ago

The images is ready. Can you please test it?

thanks for the patience

zcrself commented 5 years ago

Now I test it

On Tue, Apr 9, 2019 at 9:43 PM Luigi Faino notifications@github.com wrote:

The images is ready. Can you please test it?

thanks for the patience

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481257378, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuDgOcWdxZCl8AlMNrzRACGRdsA1gks5vfJkRgaJpZM4cLtZj .

zcrself commented 5 years ago

My Long sequence(PacBio data) could be downloaded via http://db.ncgr.ac.cn/doc/Long.fasta.gz I do not know how to generator the adaptor file

On Mon, Apr 8, 2019 at 4:13 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself do you know why? did you remove them? Anyway, can you just add a poly T? like 15T in a fasta file and like:

polyT TTTTTTTTTTTTTTT

and pass this file with -a option? let me know Luigi

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-480730951, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuFGuqHtwytPRDuwByiz8Ng3oWluMks5vevpAgaJpZM4cLtZj .

zcrself commented 5 years ago

I have tested the new image, The Previous errors disappear, the new error occurs. Are there any questions about my test data. I can give my test data to you! NAME DATE CREATED SIZE TYPE lorean_noIPRS.sif 2019-04-09 22:41:37(Shanghai Time) 3.31 Gb oci

On Tue, Apr 9, 2019 at 9:43 PM Luigi Faino notifications@github.com wrote:

The images is ready. Can you please test it?

thanks for the patience

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481257378, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuDgOcWdxZCl8AlMNrzRACGRdsA1gks5vfJkRgaJpZM4cLtZj .

lfaino commented 5 years ago

@zcrself it would be nice. Can you please send to me? additionally, can you send me the command that you run and some info on the system that you are using.

zcrself commented 5 years ago

Because this my first running, why it has these errors: GMAP database existed already?

MAPPING CONSENSUS ASSEMBLIES 23:38:48 09-04

GMAP MAPPING STARTED AT: 23:38:48 09-04

    ###BUILD INDEX###

GMAP database existed already: /db2/ncgr/mytest/LoReAn_annotation/run/gmap_output/test_scaffold.fa.masked.fasta.rename.fasta_GMAPindex --- skipping

On Wed, Apr 10, 2019 at 9:08 AM Smith Waterman zcrself@gmail.com wrote:

I have tested the new image, The Previous errors disappear, the new error occurs. Are there any questions about my test data. I can give my test data to you! NAME DATE CREATED SIZE TYPE lorean_noIPRS.sif 2019-04-09 22:41:37(Shanghai Time) 3.31 Gb oci

  • |# | 0 Elapsed Time: 0:00:00 Traceback (most recent call last): File "/opt/LoReAn/code/lorean.py", line 564, in main() File "/opt/LoReAn/code/lorean.py", line 506, in main args.verbose, Fflag=True) File "/opt/LoReAn/code/mapping.py", line 400, in gmap wd, Fflag, type_out, verbose) File "/opt/LoReAn/code/mapping.py", line 131, in gmap_map filename = longest_cds(list_fasta[0][1], list_fasta[1][1], verbose, working_dir, filename) File "/opt/LoReAn/code/mapping.py", line 189, in longest_cds db = gffutils.create_db(gff_file, ':memory:', merge_strategy='create_unique', keep_order=True, transform=transform) File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 1288, in create_db c.create() File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 504, in create self._populate_from_lines(self.iterator) File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 625, in _populate_from_lines raise ValueError("No lines parsed -- was an empty file provided?") ValueError: No lines parsed -- was an empty file provided?

On Tue, Apr 9, 2019 at 9:43 PM Luigi Faino notifications@github.com wrote:

The images is ready. Can you please test it?

thanks for the patience

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481257378, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuDgOcWdxZCl8AlMNrzRACGRdsA1gks5vfJkRgaJpZM4cLtZj .

lfaino commented 5 years ago

@zcrself , because LoReAn build the index in a previous step when align the raw reads to the genome.

zcrself commented 5 years ago

There are six files in my command line.

  1. http://db.ncgr.ac.cn/doc/test_sr_1.fastq.gz
  2. http://db.ncgr.ac.cn/doc/test_sr_2.fastq.gz
  3. http://db.ncgr.ac.cn/doc/test_scaffold.fa.gz
  4. http://db.ncgr.ac.cn/doc/rice_bamboo_prot.fasta.gz
  5. http://db.ncgr.ac.cn/doc/repeats.fix.bed.gz
  6. http://db.ncgr.ac.cn/doc/Long.fasta.gz

My command line is

lorean -rp repeats.fix.bed -sr test_sr_1.fastq,test_sr_2.fastq -lr Long.fasta -pr rice_bamboo_prot.fasta -sp rice test_scaffold.fa -t 4 --keep_tmp

Thanks for your wonderful work!

On Wed, Apr 10, 2019 at 1:58 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself it would be nice. Can you please send to me? additionally, can you send me the command that you run and some info on the system that you are using.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481544827, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuGR0M0eXMDdOlHC0bGqerZlFvZj0ks5vfX1zgaJpZM4cLtZj .

zcrself commented 5 years ago

My system info: OS: Ubuntu 16.04.5 LTS x86_64 RAM: 94G CPU: Intel(R) Xeon(R) CPU X5560 @ 2.80GHz CPU(s): 4 Disk： 2T RAID10 singularity version 3.1.0

On Wed, Apr 10, 2019 at 1:58 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself it would be nice. Can you please send to me? additionally, can you send me the command that you run and some info on the system that you are using.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481544827, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuGR0M0eXMDdOlHC0bGqerZlFvZj0ks5vfX1zgaJpZM4cLtZj .