search

beeware / batavia

A JavaScript implementation of the Python virtual machine.
http://pybee.org/batavia
Other
1.39k stars 424 forks source link

Several Test Suite fails for modules, structures, datatypes #596

Open Axle7XStriker opened 7 years ago

Axle7XStriker commented 7 years ago

I am a first time contributor and wanted to contribute to this project. According to the contribution guide i ran the test suite on my machine. And I found several test failures in tests.modules, tests.structures.test_generator, tests.datatypes . All these test suites i have ran on Linux Mint 18, python 3.5.2.

Am I doing something wrong? If yes, how can I solve this problem?

freakboy3742 commented 7 years ago

We run a CI system that runs the full test suite on every commit, so to the best of our knowledge there aren't any problems. However, sometimes there are local conditions and configurations that will cause problems. However, It's difficult to tell if you're doing something wrong without seeing the actual failures.

Axle7XStriker commented 7 years ago

Some of the failed tests which I encountered are the following:

======================================================================
FAIL: test_catch_exception (tests.structures.test_generator.YieldFromTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/structures/test_generator.py", line 313, in test_catch_exception
    """)
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/utils.py", line 445, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: 'Unknown opcode 69 (<69>)\n' != '>> 1\n>> 2\nERROR\n>> 3\n'
- Unknown opcode 69 (<69>)
+ >> 1
+ >> 2
+ ERROR
+ >> 3
 : Global context

======================================================================
FAIL: test_iterator (tests.structures.test_generator.YieldFromTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/structures/test_generator.py", line 269, in test_iterator
    """)
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/utils.py", line 445, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: 'Unknown opcode 69 (<69>)\n' != "['START', 0, 1, 2, 3, 4, 'STOP']\n"
- Unknown opcode 69 (<69>)
+ ['START', 0, 1, 2, 3, 4, 'STOP']
 : Global context

======================================================================
FAIL: test_pyclass_custom_throw (tests.structures.test_generator.YieldFromTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/structures/test_generator.py", line 392, in test_pyclass_custom_throw
    """)
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/utils.py", line 445, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: "START\n### EXCEPTION ###\nTypeError: 'generator'[56 chars]19\n" != "START\n0\nNone\n<class 'StopIteration'>\n"
  START
+ 0
+ None
+ <class 'StopIteration'>
- ### EXCEPTION ###
- TypeError: 'generator' object is not an iterator
-     test.py:25
-     test.py:19
 : Global context

======================================================================
FAIL: test_pyclass_subgenerator (tests.structures.test_generator.YieldFromTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/structures/test_generator.py", line 358, in test_pyclass_subgenerator
    """)
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/utils.py", line 445, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: "START\n### EXCEPTION ###\nTypeError: 'generator'[56 chars]17\n" != "START\n0\nNone\n<class 'StopIteration'>\n"
  START
+ 0
+ None
+ <class 'StopIteration'>
- ### EXCEPTION ###
- TypeError: 'generator' object is not an iterator
-     test.py:23
-     test.py:17
 : Global context

======================================================================
FAIL: test_simple_generator (tests.structures.test_generator.YieldFromTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/structures/test_generator.py", line 243, in test_simple_generator
    """)
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/utils.py", line 445, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: 'Unknown opcode 69 (<69>)\n' != '[1, 2]\n'
- Unknown opcode 69 (<69>)
+ [1, 2]
 : Global context

======================================================================
FAIL: test_throw (tests.structures.test_generator.YieldFromTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/structures/test_generator.py", line 291, in test_throw
    """)
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/utils.py", line 445, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: "### EXCEPTION ###\nTypeError: 'generator'[55 chars]:7\n" != "1\n<class 'KeyError'>\n<class 'StopIteration'>\n"
+ 1
+ <class 'KeyError'>
+ <class 'StopIteration'>
- ### EXCEPTION ###
- TypeError: 'generator' object is not an iterator
-     test.py:10
-     test.py:7
 : Global context

======================================================================
FAIL: test_throw_iterator (tests.structures.test_generator.YieldFromTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/structures/test_generator.py", line 326, in test_throw_iterator
    """)
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/utils.py", line 445, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: "### EXCEPTION ###\nTypeError: 'generator'[54 chars]:2\n" != "<class 'KeyError'>\n"
+ <class 'KeyError'>
- ### EXCEPTION ###
- TypeError: 'generator' object is not an iterator
-     test.py:5
-     test.py:2
 : Global context

======================================================================
FAIL: test_yield_before_and_after (tests.structures.test_generator.YieldFromTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/structures/test_generator.py", line 258, in test_yield_before_and_after
    """)
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/utils.py", line 445, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: 'Unknown opcode 69 (<69>)\n' != "['START', 1, 2, 'STOP']\n"
- Unknown opcode 69 (<69>)
+ ['START', 1, 2, 'STOP']
 : Global context

----------------------------------------------------------------------
Ran 22 tests in 59.348s

FAILED (failures=8)
Test failed: <unittest.runner.TextTestResult run=22 errors=0 failures=8>
error: Test failed: <unittest.runner.TextTestResult run=22 errors=0 failures=8>

There are some failed testcases in test_time which are:

======================================================================
FAIL: test_mktime_args (tests.modules.test_time.TimeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/modules/test_time.py", line 223, in test_mktime_args
    self.assertCodeExecution(test_str)
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/utils.py", line 445, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: '>>> [124 chars]n10344600.0\n>>> time.mktime((1970, 1, 5, 0, 0[262 chars].0\n' != '>>> [124 chars]n10348200.0\n>>> time.mktime((1970, 1, 5, 0, 0[262 chars].0\n'
  >>> import time
  >>> time.mktime((1999, 1, 1, 0, 0, 0, 0, 0, -1))
  915129000.0
  >>> time.mktime((1970, 5, 1, 0, 0, 0, 0, 0, 1))
- 10344600.0
?     ^^
+ 10348200.0
?     ^^
  >>> time.mktime((1970, 1, 5, 0, 0, 0, 0, 0, -1))
  325800.0
  >>> time.mktime((1970, 1, 1, 5, 0, 0, 0, 0, -1))
  -1800.0
  >>> time.mktime((1970, 1, 1, 0, 5, 0, 0, 0, -1))
  -19500.0
  >>> time.mktime((1970, 1, 1, 0, 0, 5, 0, 0, -1))
  -19795.0
  >>> time.mktime((1970, 1, 1, 0, 0, 0, 5, 0, -1))
  -19800.0
 : Global context

======================================================================
FAIL: test_mktime_dst (tests.modules.test_time.TimeTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/modules/test_time.py", line 248, in test_mktime_dst
    self.assertCodeExecution(test_str)
  File "/home/aman-anmol/Documents/OpenSourceContributions/batavia-dev/batavia/tests/utils.py", line 445, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)
AssertionError: '>>> [303 chars]\n7752600.0\ntrying month 5\n>>> time.mktime(([585 chars].0\n' != '>>> [303 chars]\n7756200.0\ntrying month 5\n>>> time.mktime(([585 chars].0\n'
  >>> import time
  trying month 1
  >>> time.mktime((1970, 1, 1, 0, 0, 0, 0, 0, 0))
  -19800.0
  trying month 2
  >>> time.mktime((1970, 2, 1, 0, 0, 0, 0, 0, 0))
  2658600.0
  trying month 3
  >>> time.mktime((1970, 3, 1, 0, 0, 0, 0, 0, 0))
  5077800.0
  trying month 4
  >>> time.mktime((1970, 4, 1, 0, 0, 0, 0, 0, 1))
- 7752600.0
?     -
+ 7756200.0
?    +
  trying month 5
  >>> time.mktime((1970, 5, 1, 0, 0, 0, 0, 0, 1))
- 10344600.0
?     ^^
+ 10348200.0
?     ^^
  trying month 6
  >>> time.mktime((1970, 6, 1, 0, 0, 0, 0, 0, 1))
- 13023000.0
?     ^^
+ 13026600.0
?     ^^
  trying month 7
  >>> time.mktime((1970, 7, 1, 0, 0, 0, 0, 0, 1))
- 15615000.0
?     ^^
+ 15618600.0
?     ^^
  trying month 8
  >>> time.mktime((1970, 8, 1, 0, 0, 0, 0, 0, 1))
- 18293400.0
?     ^^
+ 18297000.0
?     ^^
  trying month 9
  >>> time.mktime((1970, 9, 1, 0, 0, 0, 0, 0, 1))
- 20971800.0
?     ^^
+ 20975400.0
?     ^^
  trying month 10
  >>> time.mktime((1970, 10, 1, 0, 0, 0, 0, 0, 1))
- 23563800.0
?     ^^
+ 23567400.0
?     ^^
  trying month 11
  >>> time.mktime((1970, 11, 1, 0, 0, 0, 0, 0, 1))
- 26242200.0
?     ^^
+ 26245800.0
?     ^^
  trying month 12
  >>> time.mktime((1970, 12, 1, 0, 0, 0, 0, 0, 0))
  28837800.0
 : Global context

----------------------------------------------------------------------
Ran 35 tests in 122.098s

FAILED (failures=2, expected failures=2)
freakboy3742 commented 7 years ago

Ok - there are two sources of problems here.

The errors with Unknown opcode 69 (<69>) show you've found a problem with Python 3.5 support. The support for generators and yield from is new, and opcode 69 (GET_YIELD_FROM_ITER) hasn't been added. I've opened ticket #598 to track this issue. If you are able to use Python 3.4 in your test environment, you won't have this problem.

The time-related errors indicate there's a problem either with the timezone configuration on your computer, or with the way we're evaluating daylight saving time. This is a difficult one to diagnose without knowing exactly what timezone you're in, what timezone your computer is set to, and what timezone your browser is reporting.

This is an area we've had a lot of troubles with, because timezones are such a pain. I thought we'd sorted them all out, but if you're having problems... maybe we haven't. The best help you can provide here is to open the sandbox, run the following code:

set a breakpoint in modules/time.js, and tell us exactly which branches are executed, what tz_name is guessed, whether it has DST enabled, and how that compares with your actual local conditions.

freakboy3742 commented 7 years ago

Oops - apologies - I closed this accidentally.

Axle7XStriker commented 7 years ago

After running the test suite in python3.4 environment still there are some errors which are as follows:

Traceback (most recent call last):

  File "/usr/local/lib/python3.4/unittest/case.py", line 58, in testPartExecutor
    yield

  File "/usr/local/lib/python3.4/unittest/case.py", line 580, in run
    testMethod()

  File "/home/aman-anmol/Documents/OpenSourceContributions/temp/batavia/tests/utils.py", line 1029, in func
    substitutions=getattr(self, 'substitutions', SAMPLE_SUBSTITUTIONS)

  File "/home/aman-anmol/Documents/OpenSourceContributions/temp/batavia/tests/utils.py", line 1068, in assertBinaryOperation
    run_in_function=False,

  File "/home/aman-anmol/Documents/OpenSourceContributions/temp/batavia/tests/utils.py", line 445, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)

  File "/usr/local/lib/python3.4/unittest/case.py", line 800, in assertEqual
    assertion_func(first, second, msg=msg)

  File "/usr/local/lib/python3.4/unittest/case.py", line 1173, in assertMultiLineEqual
    self.fail(self._formatMessage(msg, standardMsg))

  File "/usr/local/lib/python3.4/unittest/case.py", line 645, in fail
    raise self.failureException(msg)

AssertionError: '>>> [875 chars]ss \'TypeError\'> : \'frozenset\' object does [9607 chars]\n\n' != '>>> [875 chars]ss \'IndexError\'> : cannot fit \'int\' into a[9616 chars]\n\n'

And the errors in test_time are :

Traceback (most recent call last):

  File "/usr/local/lib/python3.4/unittest/case.py", line 58, in testPartExecutor
    yield

  File "/usr/local/lib/python3.4/unittest/case.py", line 580, in run
    testMethod()

  File "/home/aman-anmol/Documents/OpenSourceContributions/temp/batavia/tests/modules/test_time.py", line 223, in test_mktime_args
    self.assertCodeExecution(test_str)

  File "/home/aman-anmol/Documents/OpenSourceContributions/temp/batavia/tests/utils.py", line 445, in assertCodeExecution
    self.assertEqual(js_out, py_out, context)

  File "/usr/local/lib/python3.4/unittest/case.py", line 800, in assertEqual
    assertion_func(first, second, msg=msg)

  File "/usr/local/lib/python3.4/unittest/case.py", line 1173, in assertMultiLineEqual
    self.fail(self._formatMessage(msg, standardMsg))

  File "/usr/local/lib/python3.4/unittest/case.py", line 645, in fail
    raise self.failureException(msg)

AssertionError: '>>> [124 chars]n10344600.0\n>>> time.mktime((1970, 1, 5, 0, 0[262 chars].0\n' != '>>> [124 chars]n10348200.0\n>>> time.mktime((1970, 1, 5, 0, 0[262 chars].0\n'

Also, please provide the code which you want me to run after opening the sandbox. There is nothing provided in the previous comment

smitar0y commented 6 years ago

I am also seeing these errors in test_time on windows with python 3. Will check to get more details.