Open nmadhire opened 9 years ago
Use of collect() in building the tokens from the list of surfaceforms. Tokentype needs to be serialized. Below logic should be modified,
val token = getSfs().collect().toList.flatMap( sf => lst.tokenizeUnstemmed(sf) )
Use of collect() in building the tokens from the list of surfaceforms. Tokentype needs to be serialized. Below logic should be modified,
val token = getSfs().collect().toList.flatMap( sf => lst.tokenizeUnstemmed(sf) )