pytest error

[khliu1238]

hi:

when running pytest, the following error occurred, without changing any single line of code/data:

========================== test session starts ===========================
platform darwin -- Python 2.7.10, pytest-3.5.0, py-1.5.3, pluggy-0.6.0
rootdir: /Users/wge/wge, inifile:
collected 9 items

wge/tests/miniwob/test_environment.py BASE URL: http://localhost:8765/
.BASE URL: http://localhost:8765/
========================================
[u'Click the button.', u'Click the button.', u'Click the button.']
[dummy: 'dummy', dummy: 'dummy', dummy: 'dummy']
FBASE URL: http://localhost:8765/
Iteration 1 / 50
Iteration 2 / 50
Iteration 3 / 50
Iteration 4 / 50
Iteration 5 / 50
Iteration 6 / 50
Iteration 7 / 50
Iteration 8 / 50
Iteration 9 / 50
Iteration 10 / 50
Iteration 11 / 50
Iteration 12 / 50
Iteration 13 / 50
Iteration 14 / 50
Iteration 15 / 50
Iteration 16 / 50
Iteration 17 / 50
Iteration 18 / 50
Iteration 19 / 50
Iteration 20 / 50
Iteration 21 / 50
Iteration 22 / 50
Iteration 23 / 50
Iteration 24 / 50
Iteration 25 / 50
Iteration 26 / 50
Iteration 27 / 50
Iteration 28 / 50
Iteration 29 / 50
Iteration 30 / 50
Iteration 31 / 50
Iteration 32 / 50
Iteration 33 / 50
Iteration 34 / 50
Iteration 35 / 50
Iteration 36 / 50
Iteration 37 / 50
Iteration 38 / 50
Iteration 39 / 50
Iteration 40 / 50
Iteration 41 / 50
Iteration 42 / 50
Iteration 43 / 50
Iteration 44 / 50
Iteration 45 / 50
Iteration 46 / 50
Iteration 47 / 50
Iteration 48 / 50
Iteration 49 / 50
Iteration 50 / 50
Average time: 0.0230975866318
SD: 0.000102856725524
.BASE URL: http://localhost:8765/
========================================
Clicking with click([4] button @ (61.0, 74.0) text=u'Click Me!' classes=[NO_CLASS])
{'n': [{u'reason': None, 'elapsed': 0.0477290153503418, u'done': False, u'raw_reward': 0, u'env_reward': 0}, {'elapsed': 0.34470105171203613, u'done': True, u'raw_reward': 1, u'env_reward': 0.9676}, {u'reason': None, 'elapsed': 0.3376748561859131, u'done': False, u'raw_reward': 0, u'env_reward': 0}]}
{'n': [{u'reason': None, 'elapsed': 0.10628294944763184, u'done': False, u'raw_reward': 0, u'env_reward': 0}, {'elapsed': 0.36610984802246094, u'done': True, u'raw_reward': 1, u'env_reward': 0.9676}, {u'reason': None, 'elapsed': 0.39337682723999023, u'done': False, u'raw_reward': 0, u'env_reward': 0}]}
{'n': [{u'reason': None, 'elapsed': 0.029836177825927734, u'done': False, u'raw_reward': 0, u'env_reward': 0}, {u'reason': None, 'elapsed': 0.05690813064575195, u'done': False, u'raw_reward': 0, u'env_reward': 0}, {u'reason': None, 'elapsed': 0.045066118240356445, u'done': False, u'raw_reward': 0, u'env_reward': 0}]}
Clicking with click([4] button @ (16.0, 97.0) text=u'Click Me!' classes=[NO_CLASS])
Clicking with click([4] button @ (34.0, 52.0) text=u'Click Me!' classes=[NO_CLASS])
{'n': [{'elapsed': 0.06305098533630371, u'done': True, u'raw_reward': 1, u'env_reward': 0.9937}, {'elapsed': 0.0776369571685791, u'done': True, u'raw_reward': 1, u'env_reward': 0.992}, {u'reason': None, 'elapsed': 0.06498408317565918, u'done': False, u'raw_reward': 0, u'env_reward': 0}]}
.BASE URL: http://localhost:8765/
========================================
.BASE URL: http://localhost:8765/
========================================
.BASE URL: http://localhost:8765/
========================================
- [1] body @ (0.0, 0.0) classes=[NO_CLASS] children=1
  |- [2] div @ (0.0, 0.0) classes=[NO_CLASS] children=1
     |- [3] div @ (0.0, 50.0) classes=[NO_CLASS] children=9
        |- [4] span @ (2.0, 52.0) text=u'diam a elementum,:' classes=[NO_CLASS]
        |- [5] input_text @ (2.0, 63.0) value= classes=[NO_CLASS]
        |- [6] button @ (2.0, 82.0) text=u'cancel' classes=[NO_CLASS]
        |- [7] span @ (2.0, 100.0) text=u'sed feugiat quisque:' classes=[NO_CLASS]
        |- [8] input_text @ (2.0, 111.0) value= classes=[NO_CLASS]
        |- [9] input_text @ (2.0, 130.0) value= classes=[NO_CLASS]
        |- [10] input_text @ (2.0, 149.0) value= classes=[NO_CLASS]
        |- [11] span @ (2.0, 168.0) text=u'pharetra pellentesqu...' classes=[NO_CLASS]
        |- [12] input_text @ (2.0, 179.0) value= classes=[NO_CLASS]
Clicking with click([5] input_text @ (2.0, 63.0) value= classes=[NO_CLASS])
Clicking with click([8] input_text @ (78.14, 92.0) value= classes=[NO_CLASS])
Clicking with click([5] input_text @ (2.0, 63.0) value= classes=[NO_CLASS])
Clicking with click([6] button @ (2.0, 82.0) text=u'cancel' classes=[NO_CLASS])
Clicking with click([9] button @ (2.0, 111.0) text=u'ok' classes=[NO_CLASS])
Clicking with click([6] button @ (2.0, 82.0) text=u'cancel' classes=[NO_CLASS])
.BASE URL: http://localhost:8765/
========================================
Clicking with click([4] button @ (20.0, 75.0) text=u'ONE' classes=[NO_CLASS])
Clicking with click([5] button @ (26.0, 87.0) text=u'TWO' classes=[NO_CLASS])
Clicking with click([4] button @ (64.0, 101.0) text=u'ONE' classes=[NO_CLASS])
Clicking with click([4] button @ (20.0, 113.0) text=u'ONE' classes=[NO_CLASS])
Clicking with click([5] button @ (16.0, 83.0) text=u'TWO' classes=[NO_CLASS])
Clicking with click([4] button @ (56.0, 164.0) text=u'ONE' classes=[NO_CLASS])
Clicking with click([4] button @ (82.0, 83.0) text=u'ONE' classes=[NO_CLASS])
Clicking with click([5] button @ (59.0, 109.0) text=u'TWO' classes=[NO_CLASS])
Clicking with click([4] button @ (115.0, 107.0) text=u'ONE' classes=[NO_CLASS])
Clicking with click([4] button @ (30.0, 163.0) text=u'ONE' classes=[NO_CLASS])
Clicking with click([5] button @ (113.0, 151.0) text=u'TWO' classes=[NO_CLASS])
Clicking with click([4] button @ (7.0, 166.0) text=u'ONE' classes=[NO_CLASS])
.BASE URL: http://localhost:8765/
========================================
Forward to Cathrine the email from Evy.
by: u'Evy'
to: u'Cathrine'
Send Bettine the information Lidia sent to you.
by: u'Lidia'
to: u'Bettine'
Please find the message by Patrizia, then send it to Hestia.
by: u'Patrizia'
to: u'Hestia'
Get Sharyl email and then forward it to Roz please.
dummy: 'dummy'
F

================================ FAILURES ================================
____________________ TestMiniWoBEnvironment.test_run _____________________

self = <wge.tests.miniwob.test_environment.TestMiniWoBEnvironment object at 0x104a17490>
env = <wge.miniwob.environment.MiniWoBEnvironment object at 0x104a173d0>

    def test_run(self, env):
        """Test reset and step."""
        print '=' * 40
        states = env.reset()
        print [x.utterance for x in states]
        assert all(x.utterance == 'Click the button.' for x in states)
        print [x.fields for x in states]
>       assert all(not x.fields for x in states)
E       assert False
E        +  where False = all(<generator object <genexpr> at 0x1048aebe0>)

wge/tests/miniwob/test_environment.py:66: AssertionError
_____________________ TestMiniWoBFields.test_fields ______________________

self = <wge.tests.miniwob.test_environment.TestMiniWoBFields object at 0x104dbf090>
env = <wge.miniwob.environment.MiniWoBEnvironment object at 0x104d5ae50>

    def test_fields(self, env):
        print '=' * 40
        # Training time
        states = env.reset()
        for state in states:
            print state.utterance
            print state.fields
            assert 'by' in state.fields.keys
            assert 'to' in state.fields.keys
            assert state.fields['by'] in state.utterance
            assert state.fields['to'] in state.utterance
        # Test time
        states = env.reset(mode='test')
        for state in states:
            print state.utterance
            print state.fields
>           assert not state.fields.keys
E           AssertionError: assert not ['dummy']
E            +  where ['dummy'] = dummy: 'dummy'.keys
E            +    where dummy: 'dummy' = MiniWoBState(utterance: u'Get Sharyl email and then forward it to Roz please.').fields

wge/tests/miniwob/test_environment.py:321: AssertionError
================== 2 failed, 7 passed in 42.99 seconds ===================

any idea what goes wrong?

[ppasupat]

When the task goal does not have any field (i.e., the task has no variation in the goal), at some point we added a "dummy" key to the goal, but we forgot to change the tests in this repo. I will create a pull request.

[ppasupat]

Should be fixed now. Would you mind checking?

[khliu1238]

thanks, it's fixed.