[Question] usage BAAI/bge-small-en-v1.5 model

[jeffrey-flo]

“I am trying to use the model from https://huggingface.co/BAAI/bge-small-en-v1.5, but I encounter an error during encoding. Do you have any suggestions?”

   E  FATAL EXCEPTION: DefaultDispatcher-worker-2
Process: com.projects.shubham0204.demo, PID: 13633
ai.onnxruntime.OrtException: Error code - ORT_RUNTIME_EXCEPTION - message: Non-zero status code returned while running Gather node. Name:'/embeddings/token_type_embeddings/Gather' Status Message: /onnxruntime_src/include/onnxruntime/core/framework/op_kernel_context.h:42 const T *onnxruntime::OpKernelContext::Input(int) const [T = onnxruntime::Tensor] Missing Input: token_type_ids

 at ai.onnxruntime.OrtSession.run(Native Method)
 at ai.onnxruntime.OrtSession.run(OrtSession.java:395)
 at ai.onnxruntime.OrtSession.run(OrtSession.java:242)
 at ai.onnxruntime.OrtSession.run(OrtSession.java:210)
 at com.ml.shubham0204.sentence_embeddings.SentenceEmbedding$encode$2.invokeSuspend(SentenceEmbedding.kt:56)
 at com.ml.shubham0204.sentence_embeddings.SentenceEmbedding$encode$2.invoke(Unknown Source:8)
 at com.ml.shubham0204.sentence_embeddings.SentenceEmbedding$encode$2.invoke(Unknown Source:4)
 at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:78)
 at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:167)
 at kotlinx.coroutines.BuildersKt.withContext(Unknown Source:1)
 at com.ml.shubham0204.sentence_embeddings.SentenceEmbedding.encode(SentenceEmbedding.kt:41)
 at com.projects.shubham0204.demo.MainActivity$predict$2$1.invokeSuspend(MainActivity.kt:183)
 at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
 at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
 at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
 at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
 at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
 at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
 at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
 at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)
 Suppressed: ai.onnxruntime.OrtException: Error code - ORT_RUNTIME_EXCEPTION - message: Non-zero status code returned while running Gather node. Name:'/embeddings/token_type_embeddings/Gather' Status Message: /onnxruntime_src/include/onnxruntime/core/framework/op_kernel_context.h:42 const T *onnxruntime::OpKernelContext::Input(int) const [T = onnxruntime::Tensor] Missing Input: token_type_ids

  at ai.onnxruntime.OrtSession.run(Native Method)
  at ai.onnxruntime.OrtSession.run(OrtSession.java:395)
  at ai.onnxruntime.OrtSession.run(OrtSession.java:242)
  at ai.onnxruntime.OrtSession.run(OrtSession.java:210)
  at com.ml.shubham0204.sentence_embeddings.SentenceEmbedding$encode$2.invokeSuspend(SentenceEmbedding.kt:56)
  at com.ml.shubham0204.sentence_embeddings.SentenceEmbedding$encode$2.invoke(Unknown Source:8)
  at com.ml.shubham0204.sentence_embeddings.SentenceEmbedding$encode$2.invoke(Unknown Source:4)
  at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:78)
  at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:167)
  at kotlinx.coroutines.BuildersKt.withContext(Unknown Source:1)
  at com.ml.shubham0204.sentence_embeddings.SentenceEmbedding.encode(SentenceEmbedding.kt:41)
  at com.projects.shubham0204.demo.MainActivity$predict$2$2.invokeSuspend(MainActivity.kt:185)
  ... 8 more
 Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@3114ea, Dispatchers.Default]

[shubham0204]

@jeffrey-flo Some embeddings models also need token_type_ids as inputs to the ONNX model. Currently, the native libs do not provide token_type_ids, and hence the crash. The fix should be easy and I'll try working on it from today.

The initial scope of the project was to provide a port of the all-MiniLM-L6-V2 model only, but after resolving this issue, the project will be able to support more embedding models.

[jeffrey-flo]

@shubham0204 Super! Thanks for your great works!

[shubham0204]

@jeffrey-flo You can try using the app in the 3-support-more-embedding-models branch which now supports the bge-small-en-v1.5 model. Follow up question: Are there anymore embedding models whose support would be beneficial for the project?

[jeffrey-flo]

@shubham0204 I checked the 3-support-more-embedding-models branch It's works fine !

Regards