Closed briango28 closed 5 months ago
Previous version used keras.KerasTensor
which apparently did not exist in keras v.2.
Updated to use keras.Input
instead.
Ran format.sh
.
I was working behind a MITM proxy without a proper linux environment, and had to resort to manual copying which turned out to be rather unwieldy.
Hopefully will pass tests now.
Applied above discussions. The function now looks like this:
class TokenizerWithVocabulary:
def compute_output_spec(self, input_spec) -> keras.KerasTensor:
return keras.KerasTensor(
input_spec.shape + (self.sequence_length,), dtype=self.compute_dtype
)
Thank you!
Small fix for Issue 1522
Implements the same
compute_output_spec()
method forBytePairTokenizer
,WordPieceTokenizer
, andSentencePieceTokenizer
.