Facing Issue while running `run_tf_multiple_choice.py` from examples

bhadreshpsavani commented 3 years ago

Environment info

transformers version: 4.4.0.dev0
Platform: Colab
Python version: NA
PyTorch version (GPU?): NA
Tensorflow version (GPU?):
Using GPU in script?: No
Using distributed or parallel set-up in script?: No

Who can help

Information

I was trying to train bert-base-cased on Multiple Choice task with below script provided on Readme of the task

export SWAG_DIR=/path/to/swag_data_dir
python ./examples/multiple-choice/run_tf_multiple_choice.py \
--task_name swag \
--model_name_or_path bert-base-cased \
--do_train \
--do_eval \
--data_dir $SWAG_DIR \
--learning_rate 5e-5 \
--num_train_epochs 3 \
--max_seq_length 80 \
--output_dir models_bert/swag_base \
--per_gpu_eval_batch_size=16 \
--per_device_train_batch_size=16 \
--gradient_accumulation_steps 2 \
--overwrite_output

I got below error

Invalid argument: ValueError: `generator` yielded an element of shape (4, 1, 80) where an element of shape (None, None) was expected.
Traceback (most recent call last):

  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/script_ops.py", line 249, in __call__
    ret = func(*args)

  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py", line 620, in wrapper
    return func(*args, **kwargs)

  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 938, in generator_py_func
    (ret_array.shape, expected_shape))

ValueError: `generator` yielded an element of shape (4, 1, 80) where an element of shape (None, None) was expected.

The problem arises when using:

[x] the official example scripts: (give details below)
[ ] my own modified scripts: (give details below)

The tasks I am working on is:

[x] an official GLUE/SQUaD task: (give the name) swag
[ ] my own task or dataset: (give details below)

To reproduce

Steps to reproduce the behavior:

Follow this colab Notebook to run the script and reproduce the issue.

Expected behavior

It should execute the script and train the model without the given error.

Note: The colab notebook given in the readme is not working, It's outdated maybe! I removed --logging-dir logs \ from the script because it was giving me another error

Tagging SMEs: @LysandreJik @jplu

bhadreshpsavani commented 3 years ago

Even for the run_tf_squad.py script, I am facing the issue.

Here is the colab notebook with issue and Traceback logs

Is there anything else I need to use while running the script?

jplu commented 3 years ago

Hello!

The multiple choice example needs to be reworked. A PR to fix the squad example is available https://github.com/huggingface/transformers/pull/10275. Be aware that some arguments are not implemented on the TF side.

The TF examples are under rework and should become more reliable in a near future.

bhadreshpsavani commented 3 years ago

I am closing this issue since it is already WIP

huggingface / transformers