search

tensorflow / similarity

TensorFlow Similarity is a python package focused on making similarity learning quick and easy.
Apache License 2.0
1.01k stars 104 forks source link

Problem using the "brute_force" search method #337

Open tonylincon1 opened 1 year ago

tonylincon1 commented 1 year ago

Hi, I had a problem using the "brute_force" search method, the following error occurred:

RuntimeError Traceback (most recent call last) in <cell line: 2>() 1 save_path = "/content/drive/MyDrive/BR Busca Inteligente/models/model_16/" ----> 2 model.save(save_path, save_index=True)

3 frames /content/similarity/tensorflow_similarity/models/similarity_model.py in save(self, filepath, save_index, compression, overwrite, include_optimizer, save_format, signatures, options, save_traces) 835 836 if hasattr(self, "_ix") and self._index and save_index: --> 837 self.save_index(filepath, compression=compression) 838 else: 839 msg = "The index was not saved with the model."

/content/similarity/tensorflow_similarity/models/similarity_model.py in save_index(self, filepath, compression) 779 """ 780 index_path = Path(filepath) / "index" --> 781 self._index.save(index_path, compression=compression) 782 783 def save(

/content/similarity/tensorflow_similarity/indexer.py in save(self, path, compression) 718 719 self.kv_store.save(path, compression=compression) --> 720 self.search.save(path) 721 722 @staticmethod

/content/similarity/tensorflow_similarity/search/nmslib_search.py in save(self, path) 178 fname = self.__make_fname(path) 179 tmpidx = os.path.join(tmpdirname, os.path.basename(path)) --> 180 self._search_index.saveIndex(tmpidx, save_data=True) 181 182 # read search_index.bin tmp file and write to fname

RuntimeError: SaveIndex is not implemented for method: Sequential search

I tried to follow the tutorial visualization pattern. Is this something that hasn't been implemented yet?

I used TensorFlow: 2.9.0 and TensorFlow Similarity 0.17.1

tonylincon1 commented 1 year ago

something news?

owenvallis commented 1 year ago

Hi Tony, this is a missing implementation in the nmslib package. They don't currently support saving a brute force index.

Having said that... we have a new release coming in v0.18 on the development branch. This adds support for a new in memory linear index that you can serialize and save without needing to use the nmslib deps. See here: tensorflow_similarity/search/linear_search.py

tonylincon1 commented 1 year ago

It's sounds good. I will check this implementation.

My suggestion is remove brute force in supervised tutorial that's show the implementation.

Here image

Thanks for solution xD