Training model with LanguageModelFeaturizer results in ValueError

[kearnsw]

Rasa version: 2.5.1

Rasa SDK version (if used & relevant): 2.5.0

Rasa X version (if used & relevant): None

Python version: 3.6.12

Operating system (windows, osx, ...): OSX

Issue:

Cannot train pipeline with LanguageModelFeaturizer on latest releases.

Error (including full traceback):

2021-04-29 13:45:12 INFO     rasa.nlu.model  - Starting to train component LanguageModelFeaturizer
Traceback (most recent call last):
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/bin/rasa", line 8, in <module>
    sys.exit(main())
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/__main__.py", line 117, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/cli/train.py", line 59, in <lambda>
    train_parser.set_defaults(func=lambda args: run_training(args, can_exit=True))
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/cli/train.py", line 103, in run_training
    finetuning_epoch_fraction=args.epoch_fraction,
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/api.py", line 124, in train
    loop,
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/utils/common.py", line 310, in run_in_loop
    result = loop.run_until_complete(f)
  File "uvloop/loop.pyx", line 1456, in uvloop.loop.Loop.run_until_complete
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/model_training.py", line 119, in train_async
    finetuning_epoch_fraction=finetuning_epoch_fraction,
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/model_training.py", line 300, in _train_async_internal
    finetuning_epoch_fraction=finetuning_epoch_fraction,
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/model_training.py", line 343, in _do_training
    finetuning_epoch_fraction=finetuning_epoch_fraction,
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/model_training.py", line 766, in _train_nlu_with_validated_data
    **additional_arguments,
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/nlu/train.py", line 116, in train
    interpreter = trainer.train(training_data, **kwargs)
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/nlu/model.py", line 212, in train
    component.train(working_data, self.config, **context)
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/nlu/featurizers/dense_featurizer/lm_featurizer.py", line 815, in train
    batch_docs = self._get_docs_for_batch(batch_messages, attribute)
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/nlu/featurizers/dense_featurizer/lm_featurizer.py", line 768, in _get_docs_for_batch
    batch_token_ids, batch_tokens, batch_examples, attribute, inference_mode
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/nlu/featurizers/dense_featurizer/lm_featurizer.py", line 686, in _get_model_features_for_batch
    batch_attention_mask, padded_token_ids
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/rasa/nlu/featurizers/dense_featurizer/lm_featurizer.py", line 538, in _compute_batch_sequence_features
    np.array(padded_token_ids), attention_mask=np.array(batch_attention_mask)
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 985, in __call__
    outputs = call_fn(inputs, *args, **kwargs)
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/transformers/models/bert/modeling_tf_bert.py", line 857, in call
    kwargs_call=kwargs,
  File "/Users/kearnsw/opt/anaconda3/envs/rasa2/lib/python3.6/site-packages/transformers/modeling_tf_utils.py", line 357, in input_processing
    raise ValueError(f"Data of type {type(v)} is not allowed only {allowed_types} is accepted for {k}.")
ValueError: Data of type <class 'numpy.ndarray'> is not allowed only (<class 'tensorflow.python.framework.ops.Tensor'>, <class 'bool'>, <class 'int'>, <class 'transformers.file_utils.ModelOutput'>, <class 'tuple'>, <class 'list'>, <class 'dict'>) is accepted for attention_mask.

Command or request that led to error:

rasa train

Content of configuration file (config.yml) (if relevant):

# Configuration for Rasa NLU.
# https://rasa.com/docs/rasa/nlu/components/
language: en

pipeline:
   - name: WhitespaceTokenizer
   - name: RegexFeaturizer
   - name: LexicalSyntacticFeaturizer
   - name: CountVectorsFeaturizer
   - name: CountVectorsFeaturizer
   - name: LanguageModelFeaturizer
     model_name: gpt2
     model_weights: gpt2
   - name: DIETClassifier
   - name: EntitySynonymMapper

# Configuration for Rasa Core.
# https://rasa.com/docs/rasa/core/policies/
policies:
# # No configuration for policies was provided. The following default policies were used to train your model.
# # If you'd like to customize them, uncomment and adjust the policies.
# # See https://rasa.com/docs/rasa/policies for more information.
   - name: MemoizationPolicy
   - name: TEDPolicy
     max_history: 5
     epochs: 100
   - name: RulePolicy

Content of domain file (domain.yml) (if relevant):

version: '2.0'
config:
  store_entities_as_slots: true
session_config:
  session_expiration_time: 60
  carry_over_slots_to_new_session: true
actions:
- action_1
- action_2
- action_3

Definition of done

• [ ] Investigate why this user is unable to train a model with LanguageModelFeaturizer (we know it works in our regression tests)
• [ ] Create an issue that describes the problem and how to fix (if we are able to reproduce)

[sara-tagger]

Thanks for the issue, @ancalita will get back to you about it soon!

You may find help in the docs and the forum, too 🤗

[samsucik]

Hey @kearnsw, I'm unable to reproduce the issue you're running into. What I'm doing:

1. create a clean Python 3.6.12 environment with Conda (note that I'm also on a Mac OS machine)
2. in the new environment: pip install "rasa[transformers]==2.5.1"
3. create a new project with rasa init
4. change the config in the new project to the config provided by you (ignoring the domain, which isn't used for NLU training anyway)
5. now rasa train works fine

Can you check what version of transformers you have installed? Rasa installs 2.11.0. Another thing to try would be to share some minimalistic NLU dataset on which you get the error (if you're not getting in on a fresh project like me).

[tttthomasssss]

@JEM-Mosig assigned as reviewer.

[samsucik]

@kearnsw I'll close this issue for now. Feel free to re-open if you're still experiencing the weird behaviour (especially with newer versions of Rasa Open Source).