Decode fail for numpy arrays with dtype=object

[jkgr]

Hey there, thanks to everyone involved for developing and maintaining this very useful piece of code :) The other day i tried to decode some numpy arrays and encountered an error.

~\appdata\local\continuum\miniconda2\envs\json_tricks_bug_repro\lib\site-packages\json_tricks\decoders.py in json_numpy_obj_hook(dct)
    266                 if dct['dtype'] == 'object':
    267                         dec_data = dct['__ndarray__']
--> 268                         arr = empty(dct['shape'], dtype=dct['dtype'], order=order)
    269                         for indx in ndindex(arr.shape):
    270                                 arr[indx] = nested_index(dec_data, indx)

ValueError: only 'C' or 'F' order is permitted

The error occurs for numpy arrays with parameter dtpye=object. This is a minimal example.

import numpy as np
import json_tricks
a = np.array(['a', 'b', 'c'], dtype=object)
s = json_tricks.dumps(a)
b = json_tricks.loads(s)

ValueError                                Traceback (most recent call last)
<ipython-input-24-8c3f0742c240> in <module>
      3 a = np.array(['a', 'b', 'c'], dtype=object)
      4 s = json_tricks.dumps(a)
----> 5 b = json_tricks.loads(s)

~\appdata\local\continuum\miniconda2\envs\json_tricks_bug_repro\lib\site-packages\json_tricks\nonp.py in loads(string, preserve_order, ignore_comments, decompression, obj_pairs_hooks, extra_obj_pairs_hooks, cls_lookup_map, allow_duplicates, conv_str_byte, **jsonkwargs)
    211         hooks = tuple(extra_obj_pairs_hooks) + obj_pairs_hooks
    212         hook = TricksPairHook(ordered=preserve_order, obj_pairs_hooks=hooks, allow_duplicates=allow_duplicates)
--> 213         return json_loads(string, object_pairs_hook=hook, **jsonkwargs)
    214 
    215 

...

~\appdata\local\continuum\miniconda2\envs\json_tricks_bug_repro\lib\site-packages\json_tricks\decoders.py in json_numpy_obj_hook(dct)
    266                 if dct['dtype'] == 'object':
    267                         dec_data = dct['__ndarray__']
--> 268                         arr = empty(dct['shape'], dtype=dct['dtype'], order=order)
    269                         for indx in ndindex(arr.shape):
    270                                 arr[indx] = nested_index(dec_data, indx)

ValueError: only 'C' or 'F' order is permitted

This is the piece of code raising the error:

order = 'A'
if 'Corder' in dct:
    order = 'C' if dct['Corder'] else 'F'
if dct['shape']:
    if dct['dtype'] == 'object':
        dec_data = dct['__ndarray__']
        arr = empty(dct['shape'], dtype=dct['dtype'], order=order)
        for indx in ndindex(arr.shape):
            arr[indx] = nested_index(dec_data, indx)
        return arr
    return asarray(dct['__ndarray__'], dtype=dct['dtype'], order=order)
else:
    dtype = getattr(nptypes, dct['dtype'])
    return dtype(dct['__ndarray__'])

The docs for numpy.empty state only 'C' and 'F' as options for order:

I resolved this locally by using the default order of the empty method if None is given:

order = None
if 'Corder' in dct:
    order = 'C' if dct['Corder'] else 'F'
if dct['shape']:
    if dct['dtype'] == 'object':
        dec_data = dct['__ndarray__']
        if order is None:
            arr = empty(dct['shape'], dtype=dct['dtype'])
        else:
            arr = empty(dct['shape'], dtype=dct['dtype'], order=order)
        for indx in ndindex(arr.shape):
            arr[indx] = nested_index(dec_data, indx)
        return arr
    return asarray(dct['__ndarray__'], dtype=dct['dtype'], order=order)
else:
    dtype = getattr(nptypes, dct['dtype'])
    return dtype(dct['__ndarray__'])

I'm using python 3.6.7, json_tricks-3.13.4 and numpy-1.17.4. Hopefully you can support this.

[mverleg]

Thanks for the fix! New version 3.13.5 is available