nomic-ai / nomic

Interact, analyze and structure massive text, image, embedding, audio and video datasets
https://atlas.nomic.ai
1.28k stars 168 forks source link

'SentenceTransformer' object has no attribute 'get_text_embedding_batch' while using nominic_embeddings #294

Open GildeshAbhay opened 5 months ago

GildeshAbhay commented 5 months ago

Hi,

I am using nominc embedding model witihn a custom RAG solution Here is the code

embed_model = SentenceTransformer("nomic-ai/nomic-embed-text-v1", trust_remote_code=True)
Settings.embed_model = embed_model

content = document["content"] #len(content) 
content1 = content[0:100]
doc = Document(text=content1)
node_parser = HierarchicalNodeParser.from_defaults(chunk_sizes=chunk_size)
storage_context.docstore.add_documents(nodes)
index = VectorStoreIndex(nodes, storage_context=storage_context)

getting the error below

`Traceback (most recent call last):

File "C:\Users\abhay.saini\AppData\Local\Temp\ipykernel_20568\4277666257.py", line 3, in <cell line: 3> index = VectorStoreIndex(nodes, storage_context=storage_context)

File "C:\Users\abhay.saini\AppData\Local\Programs\Python\Python38\lib\site-packages\llama_index\core\indices\vector_store\base.py", line 75, in init super().init(

File "C:\Users\abhay.saini\AppData\Local\Programs\Python\Python38\lib\site-packages\llama_index\core\indices\base.py", line 94, in init index_struct = self.build_index_from_nodes(

File "C:\Users\abhay.saini\AppData\Local\Programs\Python\Python38\lib\site-packages\llama_index\core\indices\vector_store\base.py", line 308, in build_index_from_nodes return self._build_index_from_nodes(nodes, **insert_kwargs)

File "C:\Users\abhay.saini\AppData\Local\Programs\Python\Python38\lib\site-packages\llama_index\core\indices\vector_store\base.py", line 280, in _build_index_from_nodes self._add_nodes_to_index(

File "C:\Users\abhay.saini\AppData\Local\Programs\Python\Python38\lib\site-packages\llama_index\core\indices\vector_store\base.py", line 233, in _add_nodes_to_index nodes_batch = self._get_node_with_embedding(nodes_batch, show_progress)

File "C:\Users\abhay.saini\AppData\Local\Programs\Python\Python38\lib\site-packages\llama_index\core\indices\vector_store\base.py", line 141, in _get_node_with_embedding id_to_embed_map = embed_nodes(

File "C:\Users\abhay.saini\AppData\Local\Programs\Python\Python38\lib\site-packages\llama_index\core\indices\utils.py", line 138, in embed_nodes new_embeddings = embed_model.get_text_embedding_batch(

File "C:\Users\abhay.saini\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\nn\modules\module.py", line 1695, in getattr raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")

AttributeError: 'SentenceTransformer' object has no attribute 'get_text_embedding_batch'`

Here are the versions Name: sentence-transformers Version: 2.7.0, 2.6.1 Name: llama-index Version: 0.10.24

AndriyMulyar commented 5 months ago

Please raise this issue in the sentence transformer or llama index repo.

It's not related to our API!

GildeshAbhay commented 5 months ago

can you tell me the links to their github issues page please!

msft2000 commented 3 months ago

can you tell me the links to their github issues page please!

Did you find the way to solve it? I have the same issue