yu-iskw / spark-kuromoji-tokenizer

Kuromoji Tokenizer for Spark DataFrames
https://spark-packages.org/package/yu-iskw/spark-kuromoji-tokenizer
Apache License 2.0
6 stars 2 forks source link

Update KuromojiTokenizer.scala #3

Open ashosaho opened 5 years ago

ashosaho commented 5 years ago

Spark 2.x fix. CallUDF replaced with UDF.

yu-iskw commented 5 years ago

Thanks for the PR. Can you look into the unit testing? TravisCI failed to pass the unit test. https://travis-ci.org/yu-iskw/spark-kuromoji-tokenizer

yu-iskw commented 5 years ago

@ashosaho Is there any update?