search

lfaino / LoReAn

Long Reads Annotation pipeline
MIT License
70 stars 10 forks source link

raise ValueError("No lines parsed -- was an empty file provided?") ValueError: No lines parsed -- was an empty file provided? #16

Open ncgrjsmiao opened 5 years ago

ncgrjsmiao commented 5 years ago

I test the example dataset within singularity, but met some probelm until ###GMAP MAPPING STARTED AT: 13:06:26 21-03 ###

BUILD INDEX

bellow is Error message: Traceback (most recent call last): File "/opt/LoReAn/code/lorean.py", line 552, in main() File "/opt/LoReAn/code/lorean.py", line 497, in main args.verbose, Fflag=True) File "/opt/LoReAn/code/mapping.py", line 398, in gmap wd, Fflag, type_out, verbose) File "/opt/LoReAn/code/mapping.py", line 130, in gmap_map filename = longest_cds(list_fasta[0][1], list_fasta[1][1], verbose, working_dir, filename) File "/opt/LoReAn/code/mapping.py", line 188, in longest_cds db = gffutils.create_db(gff_file, ':memory:', merge_strategy='create_unique', keep_order=True, transform=transform) File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 1288, in create_db c.create() File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 504, in create self._populate_from_lines(self.iterator) File "/usr/local/lib/python3.5/dist-packages/gffutils/create.py", line 625, in _populate_from_lines raise ValueError("No lines parsed -- was an empty file provided?") ValueError: No lines parsed -- was an empty file provided?

lfaino commented 5 years ago

@zcrself Do you use singularity exe or singularity shell?

zcrself commented 5 years ago

singularity shell

On Wed, Apr 10, 2019 at 6:23 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself Do you use singularity exe or singularity shell?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481633140, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuO1dX4udCJm37rZKRa9Xh9FztiTWks5vfbuVgaJpZM4cLtZj .

zcrself commented 5 years ago

Could you help me to run lorean using my data which I have sent to you

On Wed, Apr 10, 2019 at 7:37 PM Smith Waterman zcrself@gmail.com wrote:

singularity shell

On Wed, Apr 10, 2019 at 6:23 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself Do you use singularity exe or singularity shell?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481633140, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuO1dX4udCJm37rZKRa9Xh9FztiTWks5vfbuVgaJpZM4cLtZj .

lfaino commented 5 years ago

@zcrself already testing. I will let you now

zcrself commented 5 years ago

very appreciated!

On Wed, Apr 10, 2019 at 7:39 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself already testing. I will let you now

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481654931, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuNi8W5oYABQZT19zFZWaIFruQod1ks5vfc2JgaJpZM4cLtZj .

lfaino commented 5 years ago

@zcrself the problem was that you provided the long reads file as a ".fasta" file but actually is a fastq file. LoReAn got confused. this was the problem.

Cheers Luigi

zcrself commented 5 years ago

great! Thank you for helping me !

On Thursday, April 11, 2019, Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself the problem was that you provided the long reads file as a ".fasta" file but actually is a fastq file. LoReAn got confused. this was the problem.

Cheers Luigi

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481822358, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuBnz6fZXGTeAzyNZFPTK3P1I-DGWks5vfjYCgaJpZM4cLtZj .

zcrself commented 5 years ago

After renaming Long.fasta to Long.fastq, I will test my command line, then I will tell you the result of running

On Thu, Apr 11, 2019 at 3:05 AM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself the problem was that you provided the long reads file as a ".fasta" file but actually is a fastq file. LoReAn got confused. this was the problem.

Cheers Luigi

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481822358, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuBnz6fZXGTeAzyNZFPTK3P1I-DGWks5vfjYCgaJpZM4cLtZj .

zcrself commented 5 years ago

After renaming Long.fasta to Long.fastq, have you tested the command line. Is everything OK?

On Thu, Apr 11, 2019 at 9:37 AM Smith Waterman zcrself@gmail.com wrote:

After renaming Long.fasta to Long.fastq, I will test my command line, then I will tell you the result of running

On Thu, Apr 11, 2019 at 3:05 AM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself the problem was that you provided the long reads file as a ".fasta" file but actually is a fastq file. LoReAn got confused. this was the problem.

Cheers Luigi

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-481822358, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuBnz6fZXGTeAzyNZFPTK3P1I-DGWks5vfjYCgaJpZM4cLtZj .

lfaino commented 5 years ago

@zcrself Yes it did finish. if you restart a run, please remove the gmap and consensus folders from the run

Cheers Luigi

lfaino commented 5 years ago

@cgjosephlee @zcrself did you test the new image? did it work? can I move the noIPRS image to the production image?

i will wait your feedback

zcrself commented 5 years ago

Now I am testing the new image lfaino/lorean:noIPRS, some hours later I will tell you the result.

On Thu, Apr 11, 2019 at 3:32 PM Luigi Faino notifications@github.com wrote:

@cgjosephlee https://github.com/cgjosephlee @zcrself https://github.com/zcrself did you test the new image? did it work? can I move the noIPRS image to the production image?

i will wait your feedback

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-482001572, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuBjALHydjh7paivS_uioHEgUt5rPks5vfuUcgaJpZM4cLtZj .

zcrself commented 5 years ago

Great news.

On Thu, Apr 11, 2019 at 3:31 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself Yes it did finish. if you restart a run, please remove the gmap and consensus folders from the run

Cheers Luigi

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-482001254, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuC72IQQRnzPDpXsCdM28_xO6dpGZks5vfuTdgaJpZM4cLtZj .

cgjosephlee commented 5 years ago

I have the example run successfully. Thank you!

zcrself commented 5 years ago

Great job. I have finished my test data. Your new image is OK!

On Thu, Apr 11, 2019 at 3:32 PM Luigi Faino notifications@github.com wrote:

@cgjosephlee https://github.com/cgjosephlee @zcrself https://github.com/zcrself did you test the new image? did it work? can I move the noIPRS image to the production image?

i will wait your feedback

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-482001572, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuBjALHydjh7paivS_uioHEgUt5rPks5vfuUcgaJpZM4cLtZj .

lfaino commented 5 years ago

@cgjosephlee @zcrself @ncgrjsmiao , Can i close this issue? can you please give me a feedback on the annotation that you generate with your own data? is it any good?

Cheers Luigi

zcrself commented 5 years ago

Using test data, gff3 have generated! I have reviewed the result, it is not ideal, I think that the reason is the size of RNA-seq is small I will inscrease the size of RNA-seq， then re-run it

On Wed, Apr 17, 2019 at 1:59 PM Luigi Faino notifications@github.com wrote:

@cgjosephlee https://github.com/cgjosephlee @zcrself https://github.com/zcrself @ncgrjsmiao https://github.com/ncgrjsmiao , Can i close this issue? can you please give me a feedback on the annotation that you generate with your own data? is it any good?

Cheers Luigi

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-483948254, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuAoM7uvmN6lm9BKdMbUIJ6uSbZzYks5vhrg9gaJpZM4cLtZj .

cgjosephlee commented 5 years ago

Still had no luck on my data, maybe I'll try it again when I have time.

zcrself commented 5 years ago

Traceback (most recent call last):

File "/opt/LoReAn/code//lorean.py", line 564, in 

main()

File "/opt/LoReAn/code//lorean.py", line 188, in main

args.verbose, Fflag=True)

File "/opt/LoReAn/code/mapping.py", line 400, in gmap

wd, Fflag, type_out, verbose)

File "/opt/LoReAn/code/mapping.py", line 65, in gmap_map

record_dict = parse_fasta(reads)

File "/opt/LoReAn/code/mapping.py", line 173, in parse_fasta

with open(fasta, "r") as fh:

FileNotFoundError: [Errno 2] No such file or directory: '/db2/ncgr/maozhu/LoReAn_annotation/run/Trinity/trinity_out_dir/Trinity-GG.fasta'

Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'>

OSError: [Errno 28] No space left on device

According to this "No space left" error,

after I extend the size of my disk, Maybe I continue to run my lorean,

Need I delete some directory or files ?

On Wed, Apr 17, 2019 at 1:59 PM Luigi Faino notifications@github.com wrote:

@cgjosephlee https://github.com/cgjosephlee @zcrself https://github.com/zcrself @ncgrjsmiao https://github.com/ncgrjsmiao , Can i close this issue? can you please give me a feedback on the annotation that you generate with your own data? is it any good?

Cheers Luigi

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-483948254, or mute the thread https://github.com/notifications/unsubscribe-auth/AOSBuAoM7uvmN6lm9BKdMbUIJ6uSbZzYks5vhrg9gaJpZM4cLtZj .

lfaino commented 5 years ago

@ncgrjsmiao you can keep the AAT folder, but i would suggest to clean all.

Cheers

zcrself commented 5 years ago

In folder "/db3/ncgr/maozhu/LoReAn_annotation/run/split"

find ./ -iname "*.fasta" |wc -l

The result is "277278"

find ./ -iname "*.log" |wc -l

272696

Via 'top' command, why only one AAT is running?

Formerly the server can run many AAT jobs at the same time.

Ubuntu 16.04.5 LTS, 36cpus 72G memory 2T disk at amazonaws

On Wed, Apr 24, 2019 at 7:14 PM Luigi Faino notifications@github.com wrote:

@ncgrjsmiao https://github.com/ncgrjsmiao you can keep the AAT folder, but i would suggest to clean all.

Cheers

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-486179823, or mute the thread https://github.com/notifications/unsubscribe-auth/ADSIDOEZJYPEUUPRQIF4YNDPSA6ILANCNFSM4HBO2ZRQ .

zcrself commented 5 years ago

[image: Screen Shot 2019-04-27 at 20.53.53.png]

On Sat, Apr 27, 2019 at 9:01 PM Smith Waterman zcrself@gmail.com wrote:

In folder "/db3/ncgr/maozhu/LoReAn_annotation/run/split"

find ./ -iname "*.fasta" |wc -l

The result is "277278"

find ./ -iname "*.log" |wc -l

272696

Via 'top' command, why only one AAT is running?

Formerly the server can run many AAT jobs at the same time.

Ubuntu 16.04.5 LTS, 36cpus 72G memory 2T disk at amazonaws

On Wed, Apr 24, 2019 at 7:14 PM Luigi Faino notifications@github.com wrote:

@ncgrjsmiao https://github.com/ncgrjsmiao you can keep the AAT folder, but i would suggest to clean all.

Cheers

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-486179823, or mute the thread https://github.com/notifications/unsubscribe-auth/ADSIDOEZJYPEUUPRQIF4YNDPSA6ILANCNFSM4HBO2ZRQ .

zcrself commented 5 years ago

My command line is " lorean.py http://lorean.py/ -rp repeats.fix.bed -sr SR_1.fastq,SR_2.fastq -lr Long.fasta -pr all_pep.fasta -sp rice P_heterocycla_v1_Scaffolds_Poaceae_masked.fa -t 36 --keep_tmp 1>1.out 2>2.out &"

There are 36 CPUs on my server, now only one 'nap' job is on running What is the reason?

On Sat, Apr 27, 2019 at 9:33 PM Smith Waterman zcrself@gmail.com wrote:

[image: Screen Shot 2019-04-27 at 20.53.53.png]

On Sat, Apr 27, 2019 at 9:01 PM Smith Waterman zcrself@gmail.com wrote:

In folder "/db3/ncgr/maozhu/LoReAn_annotation/run/split"

find ./ -iname "*.fasta" |wc -l

The result is "277278"

find ./ -iname "*.log" |wc -l

272696

Via 'top' command, why only one AAT is running?

Formerly the server can run many AAT jobs at the same time.

Ubuntu 16.04.5 LTS, 36cpus 72G memory 2T disk at amazonaws

On Wed, Apr 24, 2019 at 7:14 PM Luigi Faino notifications@github.com wrote:

@ncgrjsmiao https://github.com/ncgrjsmiao you can keep the AAT folder, but i would suggest to clean all.

Cheers

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-486179823, or mute the thread https://github.com/notifications/unsubscribe-auth/ADSIDOEZJYPEUUPRQIF4YNDPSA6ILANCNFSM4HBO2ZRQ .

lfaino commented 5 years ago

@zcrself did it finish?

zcrself commented 5 years ago

No， NameError! I do not know what is the reason? How to solve it?

[image: amazon_ec2_evm_error.PNG]

On Tue, Apr 30, 2019 at 8:03 PM Luigi Faino notifications@github.com wrote:

@zcrself https://github.com/zcrself did it finish?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lfaino/LoReAn/issues/16#issuecomment-487925527, or mute the thread https://github.com/notifications/unsubscribe-auth/ADSIDOFA7VZBWK2BVICG6HTPTAYR7ANCNFSM4HBO2ZRQ .

zcrself commented 5 years ago

amazon_ec2_evm_error (1)