search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
131.71k stars 26.22k forks source link

Deebert Examples test failure #6002

Closed sshleifer closed 4 years ago

sshleifer commented 4 years ago

DeeBertTests.test_glue_deebert Excerpt from CI

2020-07-23T00:30:23.1537872Z         with patch.object(sys, "argv", train_args):
2020-07-23T00:30:23.1538109Z             result = run_glue_deebert.main()
2020-07-23T00:30:23.1538324Z             for value in result.values():
2020-07-23T00:30:23.1538573Z >               self.assertGreaterEqual(value, 0.75)
2020-07-23T00:30:23.1538856Z E               AssertionError: 0.6666666666666666 not greater than or equal to 0.75
2020-07-23T00:30:23.1539030Z
sshleifer commented 4 years ago

@Ji-Xin would you mind taking a stab at this? Command:

RUN_SLOW=1 pytest examples/deebert/