search

mverleg / pyjson_tricks

Extra features for Python's JSON: comments, order, numpy, pandas, datetimes, and many more! Simple but customizable.
Other
152 stars 22 forks source link

Use compressed numpy format by default in compressed mode (next major release) #73

Open mverleg opened 4 years ago

mverleg commented 4 years ago

Numpy arrays are stored as text arrays by default. This is readable but takes a lot of space.

When the file is compressed, it perhaps makes more sense to use the new compact format introduced in #9.

Since this is a backwards-incompatible change, it will have to wait until the next major version (4.0.0).

For now the compressed format can be used like this (compression is optional):

json = dumps(data, compression=True, properties={'ndarray_compact': True})

There's also a warning announcing this change, which you can silence by explicitly disabling the new format:

json = dumps(data, properties={'ndarray_compact': False})
mverleg commented 4 years ago

It might be important to support F-order arrays before this becomes default