philipstarkey commented 7 years ago

Original report (archived issue) by Jan Werkmann (Bitbucket: PhyNerd, GitHub: PhyNerd).

As Python 3 porting is not that far away I would like to collect python 3 incompatibilities here. And also collect fixes so they can be discussed and I don't lose them somewhere in my repos. So far besides the obvious lack of PyQt5 and unicode I found the following problems:

the module Queue is names queue in python 3 so in __main__.py we should to this:
```
#!python
if six.PY2:
import Queue
else:
import queue as Queue
```
in __main__.py there exists a line:

#!python

for column_index, name in sorted(column_names.items(), key=lambda s: s[1]):

which throws the exception:

#!python

  File "lyse/__main__.py", line 844, in populate_model
    for column_index, name in sorted(column_names.items(), key=lambda s: s[1]):
TypeError: '<' not supported between instances of 'tuple' and 'str'

A fix for this is:

#!python

        def get_key(s):
            while isinstance(s, tuple):
                s = s[1]
            return s

        for column_index, name in sorted(column_names.items(), key=get_key):

in __init__.py there are 2 unused includes one for urllib and one for urllib2 both are unused and urlliib2 doesn't exist under python 3 *in dataframe_utilities.py the function
```
#!python
```

def asdatetime(timestr): tz = tzlocal.get_localzone().zone return pandas.Timestamp(timestr, tz=tz)

doesn't work and throws the Exeption:

!python

Traceback (most recent call last): File "/Users/janwerkmann/labscript_suite/labscript_utils/excepthook/init.py", line 43, in run run_old(*args, *kwargs) File "/Users/janwerkmann/anaconda/envs/snowflakes/lib/python3.6/threading.py", line 864, in run self._target(self._args, **self._kwargs) File "/Users/janwerkmann/anaconda/envs/snowflakes/lib/python3.6/site-packages/zprocess/init.py", line 67, in f six.reraise(type, value, traceback) File "/Users/janwerkmann/anaconda/envs/snowflakes/lib/python3.6/site-packages/six.py", line 686, in reraise raise value File "lyse/main.py", line 1627, in incoming_buffer_loop dataframe = get_dataframe_from_shot(filepath) File "/Users/janwerkmann/labscript_suite/lyse/dataframe_utilities.py", line 116, in get_dataframe_from_shot nested_dict = get_nested_dict_from_shot(filepath) File "/Users/janwerkmann/labscript_suite/lyse/dataframe_utilities.py", line 60, in get_nested_dict_from_shot row['run time'] = asdatetime(h5_file.attrs['run time']) File "/Users/janwerkmann/labscript_suite/lyse/dataframe_utilities.py", line 29, in asdatetime return pandas.Timestamp(timestr, tz=tz) File "pandas/_libs/tslib.pyx", line 402, in pandas._libs.tslib.Timestamp.new (pandas/_libs/tslib.c:10051) File "pandas/_libs/tslib.pyx", line 1528, in pandas._libs.tslib.convert_to_tsobject (pandas/_libs/tslib.c:28851) TypeError: Cannot convert input [b'20170612T123722'] of type <class 'bytes'> to Timestamp

a fix for this is

!python

def asdatetime(timestr): tz = tzlocal.get_localzone().zone if isinstance(timestr, bytes): timestr = timestr.decode('utf-8') return pandas.Timestamp(timestr, tz=tz)

philipstarkey commented 7 years ago

Original comment by Jan Werkmann (Bitbucket: PhyNerd, GitHub: PhyNerd).

Edited issue description

philipstarkey commented 7 years ago

Original comment by Jan Werkmann (Bitbucket: PhyNerd, GitHub: PhyNerd).

Edited issue description

philipstarkey commented 7 years ago

Original comment by Jan Werkmann (Bitbucket: PhyNerd, GitHub: PhyNerd).

after fixing these things and with the changes from @chrisjbillington 's unicode branch everything is working smoothly no bugs encountered so far.

philipstarkey commented 7 years ago

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

I ran an automated python 2 -> python 2/3 tool, and it had very little to say about lyse, only the queue import. The other changes required therefore are "semantic" ones where automated tools can't tell what our intentions were and so need to be looked at manually, as you've seen.

Since it's obvious and completely backward-compatible I'll add the queue import to the other pull request, and the remaining, more semantic changes that you're listing here can be the subject of another pull request for Python 3 compatibility.

philipstarkey commented 7 years ago

Original comment by Chris Billington (Bitbucket: cbillington, GitHub: chrisjbillington).

The semantics of the replacement sort key are not quite right. But I've put one that is right into the pull request. I was going to hold off, but was encouraged by the fact that all the pending pull requests are all still non-conflicting, and these changes aren't so big.

The asdatetime() thing, that's because BLACS is saving a bytestring instead of a unicode string for h5_file.attrs['run time']. Whilst we could 'fix' it in BLACS, we should be backward compatible with files before the fix. So we should probably just leave that part of BLACS be - that will become a unicode string in Python 3 anyway. So I think your fix is the right approach and I've included it in the pull request.

philipstarkey commented 6 years ago

Original comment by Jan Werkmann (Bitbucket: PhyNerd, GitHub: PhyNerd).

changed state from "new" to "resolved"

lyse is Python 3 compatible since a30bc72be263672a23f36ba52825b160d5b19377

labscript-suite-temp-2 / lyse

Python 3 #30

!python

!python