Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/spark/python/pyspark/ml/util.py", line 353, in load
return cls.read().load(path)
File "/usr/lib/spark/python/pyspark/ml/util.py", line 365, in read
return JavaMLReader(cls)
File "/usr/lib/spark/python/pyspark/ml/util.py", line 296, in init
self._jread = self._load_java_obj(clazz).read()
TypeError: 'JavaPackage' object is not callable
Hi can you please help me loading the model . I am using pyspark --packages 'com.github.jelmerk:hnswlib-spark_3.3_2.12:1.0.1' command
creating instance
hnsw = HnswSimilarity(identifierCol='id', queryIdentifierCol='id' , featuresCol='company_industry_embeddings', distanceFunction='cosine', m=48, ef=5, k=200, efConstruction=200, numPartitions=50, excludeSelf=True)
test = hnsw.fit(df)
test.save(path)
reading the model
model = hnsw.load(path)
## gives error