towards python 3 support

[hayd]

add a with_metaclass top level function which allows metaclass to work syntactically in both python 2 and python 3.

don't test the python 3 only syntax test file with python 2.

---

The remaining failing tests are contracts.interface.ContractSyntaxErrors. It's unclear why these are failing, they appear to be stateful (as the code works correctly outside of the context of the test suite!). Could this be caching related?

Note: Sometimes the tests do pass on python 3.3 and 3.4!!

[hayd]

(I had to move a couple of imports to avoid circular imports... that could also be solved by a with_meta.py)

I'm very suprised that python 3.2 passes the tests and 3.3+ doesn't (or at least they don't most of the time 3.3+ fail, they seem to intermittently pass - and the things which they fail on appear to always work in isolation). I don't understand why the ContractSyntaxErrors happen!

For example, this passed on python 3.3 but it doesn't every time! ...?!?!!

[AndreaCensi]

Thanks for this --- Python 3 support is tricky.

An important piece of information: What version of PyParsing are you using? Is it the same for 3.2 and 3.3?

intermittently pass

Heisenbug!

I will look at this closely in a couple of days. ​

[AndreaCensi]

This is why it fails in Shippable with Python 3.2:

https://gist.github.com/AndreaCensi/6896dfa442fb67b5a93a

It's a problem with PyParsing I think.

On Sat, Apr 18, 2015 at 10:22 PM, Andrea Censi censi@mit.edu wrote:

Thanks for this --- Python 3 support is tricky.

An important piece of information: What version of PyParsing are you using? Is it the same for 3.2 and 3.3?

intermittently pass

Heisenbug!

I will look at this closely in a couple of days.

Andrea Censi | LIDS / MIT | http://censi.mit.edu

[hayd]

The version of pyparse appears to be 2.0.3 on those travis bots, some fail... some pass.

Locally they always fails for me on 3.4. I should clarify when I say "in isolation", e.g. this test fails:

nosetests src/contracts/testing/test_new_contract.py:TestNewContract.test_idioms

ValueError: The given condition 'list[3](number,>=0,<=1)' does not parse cleanly: Expected "-" (at char 20), (line:1, col:21)

 line  1 >list[3](number,>=0,<=1)
                              ^
                              |
                              here or nearby

However, if run in python interpreter/ipython (with the same env):

In [1]: contracts.new_contract('color', 'list[3](number,>=0,<=1)')
Out[1]: SeparateContext(List(EqualTo(SimpleRValue(3)),And([Extension('number'), CheckOrder(None,'>=',SimpleRValue(0)), CheckOrder(None,'<=',SimpleRValue(1))])))

---

Here's the travis log (for 3.3), the code is the same (the only change is in the shippable yml):

failing: https://travis-ci.org/hayd/contracts/jobs/59082622 passing: https://travis-ci.org/hayd/contracts/jobs/59081821

---

Heisenbug!

It appears that way :( :tophat:

[hayd]

Another amusement, I have a script to get the most frequent failures. This is on python 3.4 (also pyparse 2.0.3):

(py34) % python common_test.py
 17 contracts.interface.ContractSyntaxError: Expected "-" (at char 1), (line:1, col:2)
 14 contracts.interface.ContractSyntaxError: Expected "-" (at char 2), (line:1, col:3)
 12 contracts.interface.ContractSyntaxError: Expected "-" (at char 10), (line:1, col:11)
(py34) % rm nose_output.log
(py34) % nosetests 2> nose_output.log
(py34) % python common_test.py
 47 contracts.interface.ContractSyntaxError: Expected "-" (at char 1), (line:1, col:2)
 32 contracts.interface.ContractSyntaxError: Expected "-" (at char 2), (line:1, col:3)
 25 contracts.interface.ContractSyntaxError: Expected "-" (at char 12), (line:1, col:13)
(py34) % rm nose_output.log
(py34) % nosetests 2> nose_output.log
(py34) % python common_test.py
 30 contracts.interface.ContractSyntaxError: Expected "-" (at char 1), (line:1, col:2)
 25 contracts.interface.ContractSyntaxError: Expected "-" (at char 12), (line:1, col:13)
 18 contracts.interface.ContractSyntaxError: Expected "-" (at char 2), (line:1, col:3)
(py34) % rm nose_output.log
(py34) % nosetests 2> nose_output.log
(py34) % python common_test.py
 17 contracts.interface.ContractSyntaxError: Expected "-" (at char 1), (line:1, col:2)
 14 contracts.interface.ContractSyntaxError: Expected "-" (at char 2), (line:1, col:3)
 12 contracts.interface.ContractSyntaxError: Expected "-" (at char 10), (line:1, col:11)

That is, the number (and type) of test failures is changing.

[hayd]

(Note: pyparsing is 2.0.3 on all the travis machines, and python 3.2 always passes, so it must be something else...)

[AndreaCensi]

OK, here are two things to try:

1) First of all, in main.py, let _cacheable() return always False. In this way, there is nothing cached in PyContracts.

2) The only other stateful code is PyParsing's packrat I believe. Try to comment this line in src/contracts/syntax.py:

ParserElement.enablePackrat()

​

[AndreaCensi]

Just for curiosity, I checked the Python 3.2 -> 3.3 "What's new":

https://docs.python.org/3/whatsnew/3.3.html

but nothing jumps to my eyes.

[hayd]

My guess is that "Hash randomization is switched on by default." means that dicts are ordered differently... and this causes some of the seeming randomness.

Will have another play with some of the state, thanks!

[AndreaCensi]

"Hash randomization is switched on by default."

Whoa!

[hayd]

To be fair, python never guaranteed that dicts were ordered it just happened they were in cpython 2... But yes, iterating through dicts could be it!!

Removing the enablePackrat kills performance! The errors seem to persist when commenting the two things you describe above...

(Anyways, on the bright-side, this PR adds metaclass and passes more tests than before!)

[AndreaCensi]

I published v1.7.2 including this changes. Thanks again!