Attribute errors with pytorch_transformers tests

tobimichigan commented 5 years ago

🐛 Bug

Model I am using (from the official repo):

Language I am using the model on (English, Yoruba,Igbo, Hausa etc):

The problem arise when using:

[ ] the official example scripts: (give details): the test run script

The tasks I am working on is:

[ ] an official GLUE/SQUaD task: (give the name)=>yes

To Reproduce

Steps to reproduce the behavior:

1.python -m pytest -sv ./pytorch_transformers/tests/

`pytorch_transformers/tests/modeling_transfo_xl_test.py::TransfoXLModelTest::test_transfo_xl_lm_head FAILED pytorch_transformers/tests/modeling_transfo_xl_test.py::TransfoXLModelTest::test_transfo_xl_model FAILED pytorch_transformers/tests/tokenization_xlnet_test.py::XLNetTokenizationTest::test_tokenizer_no_lower PASSED

=================================== FAILURES =================================== __ TransfoXLModelTest.test_attention_outputs ___

self =

def test_attention_outputs(self):
    config, inputs_dict = self.model_tester.prepare_config_and_inputs_for_common()

    for model_class in self.all_model_classes:
        config.output_attentions = True
        config.output_hidden_states = False
        model = model_class(config)
        model.eval()

      outputs = model(**inputs_dict)

pytorch_transformers/tests/modeling_common_test.py:73:

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py:493: in call result = self.forward(*input, **kwargs) pytorch_transformers/modeling_transfo_xl.py:1253: in forward outputs = self._forward(input_ids, mems=mems, head_mask=head_mask)

self = TransfoXLModel( (word_emb): AdaptiveEmbedding( (emb_layers): ModuleList( (0): Embedding(10, 32) (1):... LayerNorm(torch.Size([32]), eps=1e-05, elementwise_affine=True) ) ) ) (pos_emb): PositionalEmbedding() ) dec_inp = tensor([[19, 69, 72, 42, 32, 34, 52, 38, 81, 71, 81, 47, 44], [22, 12, 3, 26, 63, 25, 64, 52, 79, 71, 17, 16,... [82, 26, 62, 95, 55, 79, 8, 90, 33, 83, 64, 53, 68], [ 7, 57, 63, 40, 74, 77, 50, 77, 19, 7, 53, 38, 19]]) mems = [tensor([[[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0.,... [0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.]]])] head_mask = [None, None, None, None, None]

def _forward(self, dec_inp, mems=None, head_mask=None):
    qlen, bsz = dec_inp.size()

    # Prepare head mask if needed
    # 1.0 in head_mask indicate we keep the head
    # attention_probs has shape bsz x n_heads x N x N
    # input head_mask has shape [num_heads] or [num_hidden_layers x num_heads] (a head_mask for each layer)
    # and head_mask is converted to shape [num_hidden_layers x qlen x klen x bsz x n_head]
    if head_mask is not None:
        if head_mask.dim() == 1:
            head_mask = head_mask.unsqueeze(0).unsqueeze(0).unsqueeze(0).unsqueeze(0)
            head_mask = head_mask.expand(self.n_layer, -1, -1, -1, -1)
        elif head_mask.dim() == 2:
            head_mask = head_mask.unsqueeze(1).unsqueeze(1).unsqueeze(1)
        head_mask = head_mask.to(dtype=next(self.parameters()).dtype) # switch to fload if need + fp16 compatibility
    else:
        head_mask = [None] * self.n_layer

    word_emb = self.word_emb(dec_inp)

    mlen = mems[0].size(0) if mems is not None else 0
    klen = mlen + qlen
    if self.same_length:
        all_ones = word_emb.new_ones(qlen, klen)
        mask_len = klen - self.mem_len
        if mask_len > 0:
            mask_shift_len = qlen - mask_len
        else:
            mask_shift_len = qlen
        dec_attn_mask = (torch.triu(all_ones, 1+mlen)

              + torch.tril(all_ones, -mask_shift_len)).bool()[:, :, None] # -1
E AttributeError: 'Tensor' object has no attribute 'bool'

pytorch_transformers/modeling_transfo_xl.py:1145: AttributeError

      outputs = model(**inputs)

pytorch_transformers/tests/modeling_common_test.py:185:

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py:493: in call result = self.forward(*input, **kwargs) pytorch_transformers/modeling_transfo_xl.py:1253: in forward outputs = self._forward(input_ids, mems=mems, head_mask=head_mask) E AttributeError: 'Tensor' object has no attribute 'bool'

pytorch_transformers/modeling_transfoxl.py:1145: AttributeError ____ TransfoXLModelTest.test_hidden_statesoutput ____

self =

def test_hidden_states_output(self):
    config, inputs_dict = self.model_tester.prepare_config_and_inputs_for_common()

    for model_class in self.all_model_classes:
        config.output_hidden_states = True
        config.output_attentions = False
        model = model_class(config)
        model.eval()

      outputs = model(**inputs_dict)

pytorch_transformers/tests/modeling_common_test.py:249:

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py:493: in call result = self.forward(*input, **kwargs) pytorch_transformers/modeling_transfo_xl.py:1253: in forward outputs = self._forward(input_ids, mems=mems, head_mask=head_mask)

       AttributeError: 'Tensor' object has no attribute 'bool'

pytorch_transformers/modeling_transfo_xl.py:1145: AttributeError __ TransfoXLModelTest.test_transfo_xl_lm_head __

self =

def test_transfo_xl_lm_head(self):
    self.model_tester.set_seed()
    config_and_inputs = self.model_tester.prepare_config_and_inputs()

  output_result = self.model_tester.create_transfo_xl_lm_head(*config_and_inputs)

pytorch_transformers/tests/modeling_transfo_xl_test.py:201:

pytorch_transformers/tests/modeling_transfo_xl_test.py:142: in create_transfo_xl_lm_head lm_logits_1, mems_1 = model(input_ids_1) /usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py:493: in call result = self.forward(*input, *kwargs) pytorch_transformers/modeling_transfo_xl.py:1349: in forward transformer_outputs = self.transformer(input_ids, mems=mems, head_mask=head_mask) /usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py:493: in call result = self.forward(input, **kwargs) pytorch_transformers/modeling_transfo_xl.py:1253: in forward E AttributeError: 'Tensor' object has no attribute 'bool'

pytorch_transformers/modeling_transfo_xl.py:1145: AttributeError =============================== warnings summary =============================== -- Docs: http://doc.pytest.org/en/latest/warnings.html ======= 5 failed, 206 passed, 10 skipped, 36 warnings in 171.71 seconds ======== `

Expected behavior

Seamless execution!!!

Environment

==============NVSMI LOG==============

Timestamp : Sat Aug 31 11:09:33 2019 Driver Version : 418.67 CUDA Version : 10.1

Attached GPUs : 1 GPU 00000000:00:04.0 Product Name : Tesla K80 Product Brand : Tesla

OS:Ubuntu 18.04
Python version:3.6
PyTorch version:1.1.0
PyTorch Transformers version (or branch):https://github.com/huggingface/pytorch-transformers
Using GPU ?yes
Distributed of parallel setup ?no
Any other relevant information:

Why do these attribute errors occur?