search

dlwh / epic

**Archived** Epic is a high performance statistical parser written in Scala, along with a framework for building complex structured prediction models.
http://scalanlp.org/
Apache License 2.0
469 stars 82 forks source link

Directional quotes going the wrong way for UTF close-double-quote? #41

Closed mrjf closed 9 years ago

mrjf commented 9 years ago

https://github.com/dlwh/epic/blob/master/src/main/scala/epic/preprocess/TreebankTokenizer.scala#L52

      case "“" => inOpenQuote = true; output += "``"
      ...
      case "”" => inOpenQuote = false; output += "``"

Should that be a +="''" for the curly close quote?

Thanks for Epic!

mrjf commented 9 years ago

And same for single quote?

https://github.com/dlwh/epic/blob/master/src/main/scala/epic/preprocess/TreebankTokenizer.scala#L50

dlwh commented 9 years ago

thanks!