jelmerk / hnswlib

Java library for approximate nearest neighbors search using Hierarchical Navigable Small World graphs
Apache License 2.0
260 stars 56 forks source link

Unable to load the HnswSimilarity model #55

Closed humeilPixisDatascience closed 1 year ago

humeilPixisDatascience commented 1 year ago

Hi can you please help me loading the model . I am using pyspark --packages 'com.github.jelmerk:hnswlib-spark_3.3_2.12:1.0.1' command

creating instance

hnsw = HnswSimilarity(identifierCol='id', queryIdentifierCol='id' , featuresCol='company_industry_embeddings', distanceFunction='cosine', m=48, ef=5, k=200, efConstruction=200, numPartitions=50, excludeSelf=True) test = hnsw.fit(df) test.save(path)

reading the model

model = hnsw.load(path) ## gives error

Traceback (most recent call last): File "", line 1, in File "/usr/lib/spark/python/pyspark/ml/util.py", line 353, in load return cls.read().load(path) File "/usr/lib/spark/python/pyspark/ml/util.py", line 365, in read return JavaMLReader(cls) File "/usr/lib/spark/python/pyspark/ml/util.py", line 296, in init self._jread = self._load_java_obj(clazz).read() TypeError: 'JavaPackage' object is not callable

humeilPixisDatascience commented 1 year ago

The issue is resolved now

Thanks