TomographyModel expparams_dtype broken in Python 2

[cgranade]

The Python 3 port at one point required unicode_literals to be turned on for tomography/models.py, landing us right in the fallout from numpy/numpy#2407. As a result, TomographyModel.expparams_dtype is not understood by np.empty, np.array and friends:

TypeError                                 Traceback (most recent call last)
<ipython-input-82-aeaa035a166b> in <module>()
      1 updater = qi.SMCUpdater(model, 20000, prior)
----> 2 expparams = np.empty((1,), dtype=model.expparams_dtype)

TypeError: data type not understood

Until we can fix this, a work around is to manually creating expparam arrays with field names specified as str.

[scasagrande]

If you can write me a test for this that passes in py3 but fails in py2 I'll get on it :D On May 20, 2016 8:11 AM, "Chris Granade" notifications@github.com wrote:

The Python 3 port at one point required unicode_literals to be turned on for tomography/models.py, landing us right in the fallout from numpy/numpy#2407 https://github.com/numpy/numpy/issues/2407. As a result, TomographyModel.expparams_dtype is not understood by np.empty, np.array and friends:

TypeError Traceback (most recent call last)

in () 1 updater = qi.SMCUpdater(model, 20000, prior) ----> 2 expparams = np.empty((1,), dtype=model.expparams_dtype) TypeError: data type not understood Until we can fix this, a work around is to manually creating expparam arrays with field names specified as str. — You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/QInfer/python-qinfer/issues/59

[cgranade]

Forgot to come back to this one, but a @csferrie found a simple test case today:

>>> import numpy as np
>>> import qinfer as qi
>>> basis = qi.tomography.pauli_basis(1)
>>> model = qi.tomography.TomographyModel(basis)
>>> expparams = np.empty((1,), dtype=model.expparams_dtype)
Traceback (most recent call last):
  File "<ipython-input-6-f2da9de3c693>", line 1, in <module>
    expparams = np.empty((1,), dtype=model.expparams_dtype)
TypeError: data type not understood

[cgranade]

Turns out this is a really easy bug to fix. We just need to surround field names w/ str in files that have unicode_literals turned on. Since str returns a non-Unicode string on Python 2 and a Unicode string on Python 3, this has the effect of being fully compatible with NumPy's bizarre convention for field names on the respective versions.

[scasagrande]

Hmm. I see how this fixes it, but you'll have to be careful about importing the new str implementation into these files then. If you go from builtins import str, bytes to get the new functionality into Py2.7, then this will break again.

[cgranade]

Agreed, it is a pain thanks to how futures defines newstr. Maybe it would make sense to add a function field(name, dtype, shape) that calls builtins.str instead of builtins.str or futures.str on its first argument?

On Thu, Jun 9, 2016, 23:22 Steven Casagrande notifications@github.com wrote:

Hmm. I see how this fixes it, but you'll have to be careful about importing the new str implementation into these files then. If you go from builtins import str, bytes to get the new functionality into Py2.7, then this will break again.

— You are receiving this because you authored the thread.

Reply to this email directly, view it on GitHub https://github.com/QInfer/python-qinfer/issues/59#issuecomment-224892594, or mute the thread https://github.com/notifications/unsubscribe/AAB7HEysl69Ui-eFqq7Ot7BuNSb67uq_ks5qKBOogaJpZM4IjJHz .

[scasagrande]

What is the actual requirement of these numpy functions? Can both Py2 and Py3 work with byte strings, or must it be byte for Py2 and unicode for Py3?

If both versions will accept byte strings, I'd do an from builtins import str, bytes and then make sure things are converted to bytes. But for some reason I feel that's not the case :(

[cgranade]

In the NumPy issue thread, someone suggested that, but it turns out that NumPy basically has the worst behavior in this case, and demands bytes on Py2 and stron Py3. It's... kind of bizarre.

[scasagrande]

What is this, I don't even...

The only other solution is (ugh) checking what the primary python version is at run time, and doing a

from builtin import str, bytes
if sys.version_info[0] >=3:
    str(thing)
else:
    bytes(thing)

Otherwise, carry on then...