Gemma 2b converted to tflite seems to produce random results

[AmarOk1412]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Android 14

MediaPipe Tasks SDK version

0.10.14

Task name (e.g. Image classification, Gesture recognition etc.)

Text generation

Programming Language and version (e.g. C++, Python, Java)

Java

Describe the actual behavior

Random output from .bin generated

Describe the expected behaviour

safetensors and .bin should give similar outputs

Standalone code/steps you may have used to try to get what you need

I trained a model to parse ingredients in recipes via this dataset: https://huggingface.co/datasets/AmarOk1412/ingredients

Model gives:

https://huggingface.co/AmarOk1412/fine_tuned_science_gemma2b-it

Prompt I use:

prompt = """

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
You're a bot parsing ingredients from recipes that needs to provide a structure to use in a list. The answer is a list of ingredients with optional quantrity and unit.
There can be several ingredients in the same input. This will result in a list with multiple ingredients. e.g. 'sel et poivre' will give    "sel" and "poivre" as ingredients.
Remove useless words, e.g. "sel au goût", "au goût" is not a part of the ingredient and useless in the list.

### Instruction:
{}

### Input:
{}

### Response:
{}"""

e.g.

User:
### Instruction:
parse

### Input:
sel et poivre

### Response:

Model:
[{'ingredient': 'sel', 'quantity': 'None', 'unit': 'None'}, {'ingredient': 'poivre', 'quantity': 'None', 'unit': 'None'}]

Using https://github.com/NSTiwari/Gemma-on-Android (that loads the .bin generated by mediapipe), I can't get useful output, however, using the safetensors with https://huggingface.co/spaces/Aashi/NSTiwari-fine_tuned_science_gemma2b-it or in Colab seems ok.

Playbook I use to do the conversion: https://gist.github.com/AmarOk1412/edf86f72966768177b685099b76499a7#file-fine_tune_gemma_model-ipynb

I'd expect to have similar outputs for both .safetensors and .bin

Is there a way to dig more?

Other info / Complete Logs

No response

[sunlili]

Yes, I met the same problem when apply a lora fine-tuning adapter. How to prevent the fine-tuning effect from being invalid during the model file conversion?

[AmarOk1412]

Also tried to increase the size of the dataset and use previous mediapipe versions without success

[kuaashish]

Hi @AmarOk1412,

We apologize for the delayed response. To assist you better, please specify if you are testing on an emulator or a physical device. If you are encountering a crash on a physical device, kindly provide its specifications.

Thank you!!

[AmarOk1412]

I tested on some pixel emulator and I have a Google Pixel 6 too on Android 14