zarr-developers / zarr-python

An implementation of chunked, compressed, N-dimensional arrays for Python.
https://zarr.readthedocs.io
MIT License
1.54k stars 288 forks source link

structured arrays don't support datetime #632

Open ghost opened 4 years ago

ghost commented 4 years ago

For bug reports, please follow the template below. For enhancement proposals, feel free to use whatever template makes sense (major new features should be discussed in the Zarr specifications repository https://github.com/zarr-developers/zarr-specs).

Minimal, reproducible code sample, a copy-pastable example if possible

arr = np.array([('19000101', 123), ('19021201', 321)], dtype=[('date', 'datetime64[D]'), ('num', 'i8')])
za.array(arr)
lib/python3.8/base64.py in b64encode(s, altchars)
     56     application to e.g. generate url or filesystem safe Base64 strings.
     57     """
---> 58     encoded = binascii.b2a_base64(s, newline=False)
     59     if altchars is not None:
     60         assert len(altchars) == 2, repr(altchars)

ValueError: cannot include dtype 'M' in a buffer

Version and installation information

Please provide the following:

rabernat commented 4 years ago

Hi @untoreh! Sorry no one has replied to your issue yet. It has been a very busy period for most Zarr devs.

It looks like we don't support datetimes in recarrays. One workaround would be to create two separate arrays. This works:

arr = np.array(['19000101', '19021201'], dtype='datetime64[D]')

You might also consider going through xarray, which has advanced support for datetime encoding / decoding.

This could be related to a numpy issue: https://github.com/numpy/numpy/issues/4983

@jakirkham tends to be our dtype guru. Perhaps he has a suggestion?

oguejioforO commented 2 years ago

@joshmoore, I am Oguejiofor.O.Esther an Outreachy applicant, will like to work on this issue

oguejioforO commented 2 years ago

Hi Sanket Verma, I would like you to go through the line of code I wrote in relation to structured arrays don't support datetime #632 and give me corrections concerning it. Attached below is a screenshot. [image: array.png]

On Wed, Oct 19, 2022 at 1:15 PM Sanket Verma @.***> wrote:

Assigned #632 https://github.com/zarr-developers/zarr-python/issues/632 to @oguejioforO https://github.com/oguejioforO.

— Reply to this email directly, view it on GitHub https://github.com/zarr-developers/zarr-python/issues/632#event-7620951038, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3QWLQ2YX7WPKDGVNA2OKMDWD7Q75ANCNFSM4SKJ7A4A . You are receiving this because you were assigned.Message ID: @.***>