chromadb.errors.InvalidDimensionException when saving.

Orion120 commented 9 months ago

I'm really enjoying working with LetMeDoIt. I was playing with the config file and since I've been getting the following. Is there a way to work with chromadb while running LetMeDoIt or can it only be worked on when LetMeDoIt is down? Ie. Change dimensions?



  File "/apps/myhand/bin/myhand", line 8, in <module>
    sys.exit(main())
  File "/apps/myhand/lib/python3.10/site-packages/myhand/main.py", line 256, in main
    LetMeDoItAI().startChats()
  File "/apps/myhand/lib/python3.10/site-packages/myhand/utils/assistant.py", line 1666, in startChats
    userInput = userInputLower = self.runActions("...", userInput)
  File "/apps/myhand/lib/python3.10/site-packages/myhand/utils/assistant.py", line 881, in runActions
    self.actions[feature][-1]()
  File "/apps/myhand/lib/python3.10/site-packages/myhand/utils/assistant.py", line 128, in <lambda>
    ".save": ("save content", lambda: self.saveChat(config.currentMessages)),
  File "/apps/myhand/lib/python3.10/site-packages/myhand/utils/assistant.py", line 1473, in saveChat
    config.save_chat_record(timestamp, order, i)
  File "/apps/myhand/lib/python3.10/site-packages/myhand/plugins/search chat records.py", line 53, in save_chat_record
    add_vector(collection, content, metadata)
  File "/apps/myhand/lib/python3.10/site-packages/myhand/plugins/search chat records.py", line 31, in add_vector
    collection.add(
  File "/apps/myhand/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 168, in add
    self._client._add(ids, self.id, embeddings, metadatas, documents, uris)
  File "/apps/myhand/lib/python3.10/site-packages/chromadb/telemetry/opentelemetry/__init__.py", line 127, in wrapper
    return f(*args, **kwargs)
  File "/apps/myhand/lib/python3.10/site-packages/chromadb/api/segment.py", line 375, in _add
    self._validate_embedding_record(coll, r)
  File "/apps/myhand/lib/python3.10/site-packages/chromadb/telemetry/opentelemetry/__init__.py", line 127, in wrapper
    return f(*args, **kwargs)
  File "/apps/myhand/lib/python3.10/site-packages/chromadb/api/segment.py", line 799, in _validate_embedding_record
    self._validate_dimension(collection, len(record["embedding"]), update=True)
  File "/apps/myhand/lib/python3.10/site-packages/chromadb/telemetry/opentelemetry/__init__.py", line 127, in wrapper
    return f(*args, **kwargs)
  File "/apps/myhand/lib/python3.10/site-packages/chromadb/api/segment.py", line 814, in _validate_dimension
    raise InvalidDimensionException(
chromadb.errors.InvalidDimensionException: Embedding dimension 1536 does not match collection dimensionality 768```

eliranwong commented 9 months ago

chromadb.errors.InvalidDimensionException: Embedding dimension 1536 does not match collection dimensionality

Have you changed the embedding model in the config? If you change the embedding model, old chromadb files that use a different embedding model does not work. You need to delete the old chromadb files to work with the newly chosen embedding model. For example, delete folders "memory", "chats" & "autogen" in storage directory ~/letmedoit/

Remarks: please quit the LetMeDoIt AI app first, before you delete the old chromadb files

Orion120 commented 9 months ago

Thank you for your quick reply.

eliranwong / letmedoit

chromadb.errors.InvalidDimensionException when saving. #46