Closed aburkov closed 2 years ago
You are missing info on the template, please complete the remaining. We require more info and step to have the exact same situation to reproduce this. (this is not a normal error on any of the environment we have tested)
Spark-nlp uses Kyro serialization. But if you don't make any configuration, it will be considered as defaultSerialization. You can find it in the Kryo source code as follows:
def canUseKryo(ct: ClassTag[_]): Boolean = { primitiveAndPrimitiveArrayClassTags.contains(ct) || ct == stringClassTag }
def getSerializer(ct: ClassTag[_]): Serializer = { if (canUseKryo(ct)) { cryoSerializer } else { defaultSerializer } }
Therefore, specifying "spark.serializer: org.apache.spark.serializer.KryoSerializer" while configuring spark solved the problem for me.
Spark NLP version: 3.1.1 Apache Spark version: 2.4.7
This issue is stale because it has been open 120 days with no activity. Remove stale label or comment or this will be closed in 5 days
Running a quick-start example and has this exception
Description
Running this example:
Expect the output: Output: ['Mona Lisa', 'Leonardo', 'Louvre', 'Paris']
Get this instead:
Your Environment