gnames / gnparser

GNparser normalises scientific names and extracts their semantic elements.
MIT License
39 stars 4 forks source link

Stream waiting for 50,000 lines before producing output #142

Closed LocoDelAssembly closed 3 years ago

LocoDelAssembly commented 3 years ago

Not sure if intended or not but for cases in scenarios where piping is used with a process that keeps stdout open for future use sending fewer than 50,000 names results in no output. Also, although -b1 should be ignored it isn't actually.

hernan@9900k:~/Projects/LocoDelAssembly/biodiversity/ext$ wc test-* -l
 25000 test-25000.txt
 50000 test-50000.txt
 75000 total
hernan@9900k:~/Projects/LocoDelAssembly/biodiversity/ext$ cat test-50000.txt | ./gnparser-linux --stream | wc -l
2021/02/09 18:20:39 Parsing 50000-th line
50001
hernan@9900k:~/Projects/LocoDelAssembly/biodiversity/ext$ cat test-25000.txt - | ./gnparser-linux --stream
Id,Verbatim,Cardinality,CanonicalStem,CanonicalSimple,CanonicalFull,Authorship,Year,Quality
Jivarus alienus
^C

(Last example I type "Jivarus alienus ", but since it would be the 25001th name nothing happens)

$ cat | ./gnparser-linux --stream -b1
Id,Verbatim,Cardinality,CanonicalStem,CanonicalSimple,CanonicalFull,Authorship,Year,Quality
Jivarus alienus
2021/02/09 18:23:02 Parsing 1-th line
011670bb-dcde-58d1-8a7a-d207db0982e2,Jivarus alienus,2,Jivarus alien,Jivarus alienus,Jivarus alienus,,,1

(-b1 honored, not sure if --stream ignored internally)

dimus commented 3 years ago

does the binary support --stream flag? Does it show it with 'gnparser-linux -h'?

dimus commented 3 years ago

I was able to reproduce it, so yes, something is wrong