I am using Spacy in a pyspark dataframe in a UDF. Regular Spacy works fine, but with spacy-stanza I get a 'pickling error', which suggests to me that the library is not serializable.
Here's the error: [PicklingError: Could not serialize object: TypeError: can't pickle _thread.RLock objects
I am using Spacy in a pyspark dataframe in a UDF. Regular Spacy works fine, but with spacy-stanza I get a 'pickling error', which suggests to me that the library is not serializable. Here's the error: [PicklingError: Could not serialize object: TypeError: can't pickle _thread.RLock objects