[Bug]: Error executing model: Error computing NN outputs.

[varkeyvincentindia]

What happened?

While trying to run a basic script mentioned below I'm seeing an issue.

import chromadb chroma_client = chromadb.Client()

switch create_collection to get_or_create_collection to avoid creating a new collection every time

collection = chroma_client.get_or_create_collection(name="my_collection")

switch add to upsert to avoid adding the same documents every time

collection.upsert( documents=[ "This is a document about pineapple", "This is a document about oranges" ], ids=["id1", "id2"] )

results = collection.query( query_texts=["This is a query document about florida"], # Chroma will embed this for you n_results=2 # how many results to return )

print(results)

Issue: 2024-07-22 21:04:26.098904 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running CoreML_2809954737722992935_1 node. Name:'CoreMLExecutionProvider_CoreML_2809954737722992935_1_1' Status Message: Error executing model: Error computing NN outputs.

Versions

chroma-hnswlib 0.7.5, chromadb 0.5.4, MAC OS MOnterey (12.7.5), Python: 3.12.4

Relevant log output

$ python chroma_db_basics.py 

2024-07-22 21:04:26.098904 [E:onnxruntime:, sequential_executor.cc:516 ExecuteKernel] Non-zero status code returned while running CoreML_2809954737722992935_1 node. Name:'CoreMLExecutionProvider_CoreML_2809954737722992935_1_1' Status Message: Error executing model: Error computing NN outputs.
Traceback (most recent call last):
  File "/Users/varkeyvincent/Desktop/LLM_learning_journey/LLM_Journey/chroma_db_basics.py", line 5, in <module>
    collection.add(
  File "/Users/varkeyvincent/Desktop/LLM_learning_journey/LLM_Journey/my_venv/lib/python3.12/site-packages/chromadb/api/models/Collection.py", line 80, in add
    ) = self._validate_and_prepare_embedding_set(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/varkeyvincent/Desktop/LLM_learning_journey/LLM_Journey/my_venv/lib/python3.12/site-packages/chromadb/api/models/CollectionCommon.py", line 279, in _validate_and_prepare_embedding_set
    embeddings = self._embed(input=documents)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/varkeyvincent/Desktop/LLM_learning_journey/LLM_Journey/my_venv/lib/python3.12/site-packages/chromadb/api/models/CollectionCommon.py", line 568, in _embed
    return self._embedding_function(input=input)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/varkeyvincent/Desktop/LLM_learning_journey/LLM_Journey/my_venv/lib/python3.12/site-packages/chromadb/api/types.py", line 211, in __call__
    result = call(self, input)
             ^^^^^^^^^^^^^^^^^
  File "/Users/varkeyvincent/Desktop/LLM_learning_journey/LLM_Journey/my_venv/lib/python3.12/site-packages/chromadb/utils/embedding_functions/onnx_mini_lm_l6_v2.py", line 200, in __call__
    return cast(Embeddings, self._forward(input).tolist())
                            ^^^^^^^^^^^^^^^^^^^^
  File "/Users/varkeyvincent/Desktop/LLM_learning_journey/LLM_Journey/my_venv/lib/python3.12/site-packages/chromadb/utils/embedding_functions/onnx_mini_lm_l6_v2.py", line 143, in _forward
    model_output = self.model.run(None, onnx_input)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/varkeyvincent/Desktop/LLM_learning_journey/LLM_Journey/my_venv/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 220, in run
    return self._sess.run(output_names, input_feed, run_options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running CoreML_2809954737722992935_1 node. Name:'CoreMLExecutionProvider_CoreML_2809954737722992935_1_1' Status Message: Error executing model: Error computing NN outputs.

[tazarov]

@varkeyvincentindia, thanks for reporting this. This would appear to be onnxruntime issue. Can you tell us what version do you have:

pip show onnxruntime

The last time we saw similar issues was when Onnx bumped a minor version at around 1.17.

Another issue might be CoreML.

How did you come by this error? Did you upgrade to a new configuration, or is this a green field project?

[varkeyvincentindia]

onnxruntime v1.18.1., coremltools v7.2. Yes, this is a green field project.

[varkeyvincentindia]

@tazarov Any update on the fix ?

[marekchen]

same issue. macos with intel chip

[oharlem]

same, macos, Intel, from under conda (microconda)

[nhanngh]

same issue. macos with intel chip

[tazarov]

@varkeyvincentindia, @oharlem, @nhanngh I did some further digging and it seems that there are cases where CoreML provider for the Onnx Runtime might not be properly supported. To resolve this you can use CPU provider as follows:

from chromadb.utils.embedding_functions.onnx_mini_lm_l6_v2 import ONNXMiniLM_L6_V2 # this is how you import in Chroma 0.5.0+
# from chromadb.utils.embedding_functions import ONNXMiniLM_L6_V2 # legacy import for Chroma <=0.4.24

ef = ONNXMiniLM_L6_V2(preferred_providers=["CPUExecutionProvider"])
import chromadb
client = chromadb.Client()
collection = client.get_or_create_collection("<collection_name>", embedding_function=ef)
collection.upsert(
documents=[
"This is a document about pineapple",
"This is a document about oranges"
],
ids=["id1", "id2"]
)

The above code will create a collection with the default embedding function with the CPU provider.