tensorflow / text

Making text a first-class citizen in TensorFlow.
https://www.tensorflow.org/beta/tutorials/tensorflow_text/intro
Apache License 2.0
1.22k stars 335 forks source link

How to use my own data? #1004

Open Westeford opened 2 years ago

Westeford commented 2 years ago

I copied the code and got this working on my computer, but when I try to change the path to file, and give it something other than the example text, it crashes.

Default (This one always works) path_to_file = tf.keras.utils.get_file('shakespeare.txt', 'https://storage.googleapis.com/download.tensorflow.org/data/shakespeare.txt')

What I want: path_to_file = tf.keras.utils.get_file('mytexttxt', 'file:///C:/Users/my_path/mytext.txt')

I am confused, what am I missing? rnn.zip

broken commented 2 years ago

What's the error? Can it not load the file? parse it? other?

Westeford commented 2 years ago

`Traceback (most recent call last): File "C:\Users\path\python\rnn\fresh\Start.py", line 220, in next_char, states = one_step_model.generate_one_step(next_char, states=states) File "C:\Users\path\AppData\Local\Programs\Python\Python310\lib\site-packages\tensorflow\python\util\traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "C:\Users\path\AppData\Local\Programs\Python\Python310\lib\site-packages\tensorflow\python\eager\execute.py", line 54, in quick_execute tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name, tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:

Detected at node 'strided_slice' defined at (most recent call last): File "", line 1, in File "C:\Users\path\AppData\Local\Programs\Python\Python310\lib\idlelib\run.py", line 164, in main ret = method(*args, **kwargs) File "C:\Users\path\AppData\Local\Programs\Python\Python310\lib\idlelib\run.py", line 578, in runcode exec(code, self.locals) File "C:\Users\path\python\rnn\fresh\Start.py", line 220, in next_char, states = one_step_model.generate_one_step(next_char, states=states) File "C:\Users\path\python\rnn\fresh\Start.py", line 197, in generate_one_step predicted_logits = predicted_logits[:, -1, :] Node: 'strided_slice' slice index -1 of dimension 1 out of bounds. [[{{node strided_slice}}]] [Op:__inference_generate_one_step_9444]`

broken commented 2 years ago

Specifically, that error is happening because predicted_logits isn't at least rank 2 (needs to be 3).

Broadly, this is likely happening because the text your using is problematic. Can you review your input text. Where did you get it? Is it formatted the same? Does Romeo have lines in it?