Unicode all the things

[hayd]

Especially in the web module, tests atm intermittently fail eg https://travis-ci.org/pattern3/pattern/jobs/41182906 (that commit was a pass when made PR, then failed when I merged (several days later), now it passes again).

I guess need to take care that everything is decoded as unicode once it's read in.

Would be good if we could unit test some of these so they're not intermittent!

[jburb]

Hey there, been programming for a while but I'm new to python. I'd like to help out with getting Pattern up to Python3 - is this a good issue to start on?

[hayd]

That would be awesome!

IIRC the remaining tasks are mostly unicode related... I warn you there's going to be some tricky things/subtleties here as it's a legacy project (specifically the test code intermittently failing, depending on the current search results, makes things difficult atm)! But if you're up for a challenge/learning... :)

I recommend running the tests test_web.py or test_db.py e.g. with nosetests test/test_web.py and see what the errors are (I usually debug using nosetests test/test_web.py --pdb --pdb-error and try and correct/get working these lines in the repl, then correct them in the file, and run again). Python 3 is much more sensitive for unicode so if you get it passing in python 3 that's a big win, and hopefully it should work (or, at least, it's be easier to make it work and worthy of a pull request!) in python 2.

Another task which would be useful, and may be easier (?) / smaller, is removing FIXME test skips in https://github.com/pattern3/pattern/pull/3/files - which don't even work in python 2 (!), and get them passing.

It may be worth thinking about the pipeline of unicode throughout the library, getting this clean/transparent (ensuring everything is unicode) would definitely be a big win. That may a herculean task however... cc @tom-de-smedt

[jburb]

Cool. Newbie question -- how do I run those unit tests? Just run the file they're in?

On Wed, Feb 11, 2015 at 11:34 AM, Andy Hayden notifications@github.com wrote:

That would be awesome!

IIRC the remaining tasks are mostly unicode related... I warn you there's going to be some tricky things/subtleties here as it's a legacy project (specifically the test code intermittently failing, depending on the current search results, makes things difficult atm)! But if you're up for a challenge/learning... :)

I recommend running the tests test_web.py or test_db.py e.g. with nosetests test/test_web.py and see what the errors are (I usually debug using nosetests test/test_web.py --pdb --pdb-error and try and correct/get working these lines in the repl, then correct them in the file, and run again). Python 3 is much more sensitive for unicode so if you get it passing in python 3 that's a big win, and hopefully it should work (or, at least, it's be easier to make it work and worthy of a pull request!) in python 2.

Another task which would be useful, and may be easier (?) / smaller, is removing FIXME test skips in https://github.com/pattern3/pattern/pull/3/files - which don't even work in python 2 (!), and get them passing.

It may be worth thinking about the pipeline of unicode throughout the library, getting this clean/transparent (ensuring everything is unicode) would definitely be a big win. That may a herculean task however... cc @tom-de-smedt https://github.com/tom-de-smedt

— Reply to this email directly or view it on GitHub https://github.com/pattern3/pattern/issues/12#issuecomment-73948281.

[hayd]

Running the file should work. I use nose (which prints output nicely and allows you to drop in):

$ nosetests test/test_web.py
$ nosetests test/test_web.py --pdb --pdb-error  # to drop in to failing tests

You may need to install it.

[inishchith]

Hey There , been programming in python for a while . i'd like to contribute by helping out Pattern to get upto python3 . How can successfully contribute ?