Cannot convert gpt2 generate into concrete function for tf lite

[tianhaoz95]

Describe the bug

When I want to convert a generate function to a concrete function for tf lite, it errors out with AttributeError: 'Tensor' object has no attribute 'numpy', the stack trace didn't seem to lead me to some possible cause, so want to bring it up here to see if anyone have some ideas.

To Reproduce

gpt2_preprocessor = keras_nlp.models.GPT2CausalLMPreprocessor.from_preset(
    "gpt2_base_en",
    sequence_length=256,
    add_end_token=True,
)

gpt2_lm = keras_nlp.models.GPT2CausalLM.from_preset(
    "gpt2_base_en",
    preprocessor=gpt2_preprocessor)

@tf.function
def generate(prompt, max_length):
    return gpt2_lm.generate(prompt, max_length)

concrete_func = generate.get_concrete_function(
    tf.TensorSpec([], tf.string), 20)

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[<ipython-input-7-6300d7c3b49e>](https://localhost:8080/#) in <cell line: 7>()
      5         dtype=tf.string)
      6 
----> 7 concrete_func = generate.get_concrete_function(
      8     tf.TensorSpec([], tf.string), default_max_len)

36 frames
[/usr/local/lib/python3.10/dist-packages/tensorflow/python/framework/ops.py](https://localhost:8080/#) in __getattr__(self, name)
    441         np_config.enable_numpy_behavior()
    442       """)
--> 443     self.__getattribute__(name)
    444 
    445   @staticmethod

AttributeError: in user code:

    File "<ipython-input-7-6300d7c3b49e>", line 5, in generate  *
        dtype=tf.string)
    File "/usr/local/lib/python3.10/dist-packages/keras_nlp/src/models/gpt2/gpt2_causal_lm.py", line 502, in generate  *
        return self._normalize_generate_outputs(outputs, input_is_scalar)
    File "/usr/local/lib/python3.10/dist-packages/keras_nlp/src/models/gpt2/gpt2_causal_lm.py", line 422, in normalize  *
        return tensor_to_string_list(x) if is_string else x.numpy()
    File "/usr/local/lib/python3.10/dist-packages/keras_nlp/src/utils/tf_utils.py", line 65, in tensor_to_string_list  *
        list_outputs = tensor_to_list(inputs)
    File "/usr/local/lib/python3.10/dist-packages/keras_nlp/src/utils/tf_utils.py", line 50, in tensor_to_list  *
        list_outputs = inputs.numpy()

    AttributeError: 'Tensor' object has no attribute 'numpy'

Expected behavior

Additional context

N/A

Would you like to help us fix it?

I would love to if someone can point me to more information.

[RageshAntonyHM]

@tianhaoz95 Did you fix it ?

[tianhaoz95]

@tianhaoz95 Did you fix it ?

I haven't retried after the potential fix.

[RageshAntonyHM]

@tianhaoz95

check this please

https://github.com/keras-team/keras-nlp/issues/1418#issuecomment-1947736035

[zhubarb]

@tianhaoz95

check this please

#1418 (comment)

The thread you refer to does not have a solution to this.