Closed alexott closed 3 years ago
Thanks for reporting this. The model is missing from our ResourceDownloader and we missed this in our unit tests. Will fix it in the next hot-fix release.
I would really like to change the code on my local machine before the next hot-fix release is out to experiment with the SentenceDetectorDL. However i just can't find the files you changed in the relevant pull request. Any hints where those are located?
@davidlenz If you need to test the PR, you need to:
This will give you a FAT JAR with the fix (inside target/...). Then you can start your SparkSession with that Fat JAR:
spark = SparkSession.builder \
.appName("Spark NLP")\
.master("local[4]")\
.config("spark.driver.memory","16G")\
.config("spark.driver.maxResultSize", "2G") \
.config("spark.jars", "/spark-nlp/target/scala-2.11/spark-nlp-assembly-2.6.2.jar")\
.config("spark.kryoserializer.buffer.max", "1000M")\
.getOrCreate()
Then you have the fix from that PR. No need to do anything to the PyPI spark-nlp, the fix is all on the JAR side.
Thanks a lot for the quick reply @maziyarpanahi ! I will try to do as you advised.
I have made a release with the fix just in case you want to test it:
pip install spark-nlp==2.6.3-rc1
and the sparknlp.start()
will take care of the rest or if needed we have 2.6.3-rc1
on the Maven as well
Thanks a lot! Works great for me. Really happy with the splitting results from the multilingual model so far.
Thanks a lot! Works great for me. Really happy with the splitting results from the multilingual model so far.
Thank you for your feedback, appreciate it 😃
Description
I was trying the code from the sentence detector demo, but it gives the error shown below:
Expected Behavior
example works without error
Current Behavior
code throws the error:
Steps to Reproduce
Context
just playing
Your Environment