Support for python 3

[GoogleCodeExporter]

Are you planning to provide compatibility with python 3.x ?

Thanks,

   F.

Original issue reported on code.google.com by fninocor...@gmail.com on 17 May 2013 at 12:41

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Distribute's use_2to3 flag (http://pythonhosted.org/setuptools/python3.html) is 
one simple option for this.

Original comment by keith.hu...@gmail.com on 16 Dec 2013 at 11:41

[GoogleCodeExporter]

I would prefer to bite the bullet and see what changes have to be made. I would 
hate to have to split the codebase. It would be quite unmaintainable...
Probably early in the new year, once the current release is out.

Original comment by bunbu...@gmail.com on 16 Dec 2013 at 5:48

• Changed state: Accepted

[GoogleCodeExporter]

Sounds good. I don't think it will be quite so bad. To get an idea of what the 
differences are, you can try making a copy of the code and running "2to3 -w 
<copied dir>". Looking at the diff of a couple files should give you an idea of 
where the incompatibilities lie.

I just tried using the distribute approach mentioned above on the latest 
version of Ruffus from Git and so far everything seems to work fine with Python 
3.3.3. I only tested a couple of the decorators (transform and merge), but 
didn't run into any problems.

In case it helps, I included a slightly modified version of setup.py which 
makes use of distribute instead of ezsetup.py. If you decide to go that route, 
you will probably also want to create a MANIFEST file containing, at least, 
"+include distribute_setup.py" -- otherwise distribute_setup.py won't be 
included in when running setup.py bdist, etc.

distribute_setup.py source: http://python-distribute.org/distribute_setup.py

Original comment by keith.hu...@gmail.com on 16 Dec 2013 at 6:06

Attachments:

• setup.py
• distribute_setup.py

[GoogleCodeExporter]

One more thing I forgot to mention: I commented out the ruffus_version import 
section -- because that results in a bunch of other unconverted modules 
attempting to load, it causes problems when Python3 tries to run setup.py.

Original comment by keith.hu...@gmail.com on 16 Dec 2013 at 6:09

[GoogleCodeExporter]

> I would prefer to bite the bullet and see what changes have to be made. I 
would hate to have to split the codebase. It would be quite unmaintainable...

There's no reason the codebase should need to be split, particularly if using 
2to3.  

"Setuptools also helps you to support Python 2 and Python 3 from the same 
source code by letting you run 2to3 on the code as a part of the build process, 
by setting the keyword parameter use_2to3 to True."

Original comment by j...@garrison.cc on 16 Dec 2013 at 6:27

[GoogleCodeExporter]

Here is what I did to use Ruffus 2.4 with Python 3.4.

I did a "2to3" on Ruffus' directory. I've replaced 'imap' for 'map' as 
documented elsewhere 
(http://kokoko.fluxionary.net/using-the-pipeline-module-ruffus-with-python-3-2/)
.

Nothing was committed to the sqlite database. It looks like it's the "buffer" 
function's fault. It didn't raise an exeception, but it didn't commit to the 
database. I removed the 'buffer' function in "__setitem__" of "dbdict.py" and 
cast the database value to bytes in the "__getitem__" function".

    def __getitem__(self, key):
        '''Return value for specified key'''
        row = self.con.execute('select value from data where key=?',
                               (key, )).fetchone()
        if not row:
            raise KeyError(key)
        ## return pickle.loads(str(row[0])) if self.picklevalues else row[0]
        return pickle.loads(bytes(row[0])) if self.picklevalues else row[0]

    def __setitem__(self, key, value):
        '''Set value at specified key'''
        if self.picklevalues:
            ## value = buffer(pickle.dumps(value, protocol=-1))
            value = pickle.dumps(value, protocol=-1)
        self.con.execute('insert or replace into data (key, value) '
                         'values (?,?)', (key, value))
        self.con.commit()

Looks like it works for me now, since Ruffus only runs jobs when needed.

Original comment by louis...@gmail.com on 14 Apr 2014 at 5:55

[GoogleCodeExporter]

Original comment by bunbu...@gmail.com on 14 May 2014 at 10:29

• Added labels: Priority-High, Type-Enhancement
• Removed labels: Priority-Medium, Type-Defect

[GoogleCodeExporter]

Fixes committed in version 2.5 beta
Changes:
1) sort in python3 does not order mixed types, i.e. int(), list() and string() 
are incommensurate
2) Print is a function, import from future
3) items() only returns a list in python2. Rewrite whenever this might cause a 
performance bottleneck
4) zip and map return iterators when imported from future
5) cPickle->pickle CStringIO->io need to be conditionally imported
6) Most changes in test case code.

Original comment by bunbu...@gmail.com on 22 Jul 2014 at 12:58

• Changed state: Verified