Add time series classification with Transformer model

[terrytangyuan]

Performance matches with https://keras.io/examples/timeseries/timeseries_classification_transformer/:

Epoch 1/200
45/45 [==============================] - 296s 7s/step - loss: 1.0540 - sparse_categorical_accuracy: 0.4986 - val_loss: 0.7607 - val_sparse_categorical_accuracy: 0.5548
Epoch 2/200
45/45 [==============================] - 294s 7s/step - loss: 0.8758 - sparse_categorical_accuracy: 0.5569 - val_loss: 0.6928 - val_sparse_categorical_accuracy: 0.5922
Epoch 3/200
45/45 [==============================] - 301s 7s/step - loss: 0.8041 - sparse_categorical_accuracy: 0.5892 - val_loss: 0.6479 - val_sparse_categorical_accuracy: 0.6338
Epoch 4/200
45/45 [==============================] - 299s 7s/step - loss: 0.7637 - sparse_categorical_accuracy: 0.6108 - val_loss: 0.6176 - val_sparse_categorical_accuracy: 0.6588
Epoch 5/200
45/45 [==============================] - 299s 7s/step - loss: 0.7213 - sparse_categorical_accuracy: 0.6309 - val_loss: 0.5983 - val_sparse_categorical_accuracy: 0.6852
Epoch 6/200
45/45 [==============================] - 301s 7s/step - loss: 0.6925 - sparse_categorical_accuracy: 0.6483 - val_loss: 0.5834 - val_sparse_categorical_accuracy: 0.6893
Epoch 7/200
45/45 [==============================] - 308s 7s/step - loss: 0.6674 - sparse_categorical_accuracy: 0.6486 - val_loss: 0.5723 - val_sparse_categorical_accuracy: 0.7046
Epoch 8/200
45/45 [==============================] - 311s 7s/step - loss: 0.6423 - sparse_categorical_accuracy: 0.6684 - val_loss: 0.5636 - val_sparse_categorical_accuracy: 0.6990
Epoch 9/200
45/45 [==============================] - 303s 7s/step - loss: 0.6152 - sparse_categorical_accuracy: 0.6878 - val_loss: 0.5558 - val_sparse_categorical_accuracy: 0.7018
Epoch 10/200
45/45 [==============================] - 305s 7s/step - loss: 0.6015 - sparse_categorical_accuracy: 0.6979 - val_loss: 0.5475 - val_sparse_categorical_accuracy: 0.7101
Epoch 11/200
45/45 [==============================] - 307s 7s/step - loss: 0.5787 - sparse_categorical_accuracy: 0.7125 - val_loss: 0.5403 - val_sparse_categorical_accuracy: 0.7101
Epoch 12/200
45/45 [==============================] - 307s 7s/step - loss: 0.5405 - sparse_categorical_accuracy: 0.7184 - val_loss: 0.5333 - val_sparse_categorical_accuracy: 0.7212
Epoch 13/200
45/45 [==============================] - 307s 7s/step - loss: 0.5609 - sparse_categorical_accuracy: 0.7212 - val_loss: 0.5282 - val_sparse_categorical_accuracy: 0.7254

[t-kalinowski]

When rendering this example, I get an error after training for a few epochs. At first glance it looks like there is a tensor shape mismatch somewhere:

Quitting from lines 155-188 (timeseries_classification_transformer.qmd) 

Error in py_call_impl(callable, dots$args, dots$keywords) : 
  TypeError: in user code:
<... omitted ...>mask)
    File "/home/tomasz/.virtualenvs/r-tensorflow-website/lib/python3.10/site-packages/keras/utils/metrics_utils.py", line 77, in decorated
        update_op = update_state_fn(*args, **kwargs)
    File "/home/tomasz/.virtualenvs/r-tensorflow-website/lib/python3.10/site-packages/keras/metrics/base_metric.py", line 140, in update_state_fn
        return ag_update_state(*args, **kwargs)
    File "/home/tomasz/.virtualenvs/r-tensorflow-website/lib/python3.10/site-packages/keras/metrics/base_metric.py", line 691, in update_state  **
        matches = ag_fn(y_true, y_pred, **self._fn_kwargs)
    File "/home/tomasz/.virtualenvs/r-tensorflow-website/lib/python3.10/site-packages/keras/metrics/metrics.py", line 3672, in sparse_categorical_accuracy
        if matches.shape.ndims > 1 and matches.shape[-1] == 1:

    TypeError: '>' not supported between instances of 'NoneType' and 'int'

See `reticulate::py_last_error()` for details
Calls: .main ... evaluate.keras.engine.training.Model -> do.call -> <Anonymous> -> py_call_impl
Execution halted

[t-kalinowski]

Thank you!