search

quadrismegistus / prosodic

Prosodic: a metrical-phonological parser, written in Python. For English and Finnish, with flexible language support.
http://quadrismegistus.github.io/prosodic/
GNU General Public License v3.0
277 stars 43 forks source link

Support for Unicode #10

Closed joshseides closed 5 years ago

joshseides commented 6 years ago

Hi @quadrismegistus! Really love this library. I am wondering if there is support for Unicode text (or if I am doing something wrong). I am doing this in Python 2.7:

import prosodic as p

# input_text is some string
text = p.Text(input_text)
text.parse()

I get this error when putting in this poem as text:

[2018-11-12 11:14:39,062: ERROR/ForkPoolWorker-6] raised unexpected: UnicodeDecodeError('ascii', 'O|no|IT|is|AN|ev|ER-|fix\xc3\xa8d|MARK', 24, 25, 'ordinal not in range(128)')
Traceback (most recent call last):

...

  line 16, in parseText
    text.parse()
  File "/usr/local/lib/python2.7/site-packages/prosodic/lib/Text.py", line 484, in parse
    ent.scansion(meter=meter,conscious=True)
  File "/usr/local/lib/python2.7/site-packages/prosodic/lib/Line.py", line 137, in scansion
    self.om("\t".join( [unicode(x) for x in [makeminlength(unicode(self),config['linelen']), makeminlength(unicode(bp) if bp else '', config['linelen']),meterstr,len(self.allParses(meter)),count,lowestScore,str_ot] ] ),conscious=conscious)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 24: ordinal not in range(128)
quadrismegistus commented 6 years ago

Hi there! Thanks for your patience. I think I may have fixed this. Could you refresh the code and try again? Also, major updates to prosodic. Now you can install through pip:

pip install prosodic

this will install "prosodic_data" in your home directory, where you can configure a site-packages install of prosodic. "prosodic" also becomes available as an executable. "prosodic text_or_folder" will open the interactive having loaded that text or folder.