Closed charles-cloud closed 1 month ago
Hi @charles-cloud,
Could you please let us know if you're testing on a physical device or an emulator?
Thank you!!
Hi @kuaashish I have tried on both emulator and physical device. but i got the same error with mediapipe sample code.
Hi @charles-cloud,
Apologies for the request, but could you please confirm if you are making any customizations to the codebase or following the exact steps from our documentation? If possible, could you point us to the specific details?
Thank you!!
Hi @kuaashish
I just tried the sample code mentioned in following link and set the model file path in the InferenceModel.kt file. https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference/android
git clone https://github.com/google-ai-edge/mediapipe-samples cd mediapipe git sparse-checkout init --cone git sparse-checkout set examples/llm_inference/android
Please share how Java side can send the input text (prompt message) to JNI, I have used generateResponse() API. But not able to find any input text in corresponding native LlmInferenceEngine_Session_PredictAsync() API.
void LlmInferenceEngine_Session_PredictAsync( LlmInferenceEngine_Session session, void callback_context, void (callback)(void callback_context, LlmResponseContext* response_context)) {
void start_llm_function(void args) { struct LlmInferenceEngineCpu_Session cpu_session = (struct LlmInferenceEngineCpu_Session)args;
std::vector
auto status = cpu_session->engine->tokenizer->Encode(cpu_session->prompt, &prompt_ids);
if (!status.ok()) { ABSL_LOG(FATAL) << "Failed to encode input: " << status; }
Hi @charles-cloud,
We have a newer version available 0.10.15. Could you please try it and let us know if you are still experiencing the same behavior.
Thank you!!
Hi @kuaashish
Thank you, Yes, i have already confirmed that JNI API inconsistency is solved on 0.10.15, Again i blocked with https://github.com/google-ai-edge/mediapipe/issues/5600. Could you please help on this issue.
Hi @charles-cloud,
Thank you for the confirmation. We are closing this issue and marking it as resolved internally. We will also review issue #5600 and provide you with an update.
[]()
Have I written custom code (as opposed to using a stock example script provided in MediaPipe)
None
OS Platform and Distribution
Android 13
MediaPipe Tasks SDK version
MEDIAPIPE_FULL_VERSION = "0.10.15"
Task name (e.g. Image classification, Gesture recognition etc.)
genai
Programming Language and version (e.g. C++, Python, Java)
java
Describe the actual behavior
FATAL exception
Describe the expected behaviour
App should not crash
Standalone code/steps you may have used to try to get what you need
Other info / Complete Logs