NeurodataWithoutBorders / pynwb

A Python API for working with Neurodata stored in the NWB Format
https://pynwb.readthedocs.io
Other
175 stars 85 forks source link

ObjectMapper datetime override fails #1333

Open weiglszonja opened 3 years ago

weiglszonja commented 3 years ago

Description

I'm trying to create an extension with a datetime attribute (see the snippet below), however when creating such extension with a datetime object raises an ObjectMapper error (see below). Tried extending ObjectMapper to override the default mapping for that attribute, but it still expects unicode or ascii string instead of a datetime object. This is the error message:

Error Traceback (most recent call last): File "/Users/weian/opt/anaconda3/envs/ndx-tank-metadata/lib/python3.8/site-packages/hdmf/build/objectmapper.py", line 911, in __add_attributes attr_value, attr_dtype = self.convert_dtype(spec, attr_value) File "/Users/weian/opt/anaconda3/envs/ndx-tank-metadata/lib/python3.8/site-packages/hdmf/build/objectmapper.py", line 249, in convert_dtype ret = spec_dtype_type(value) File "/Users/weian/opt/anaconda3/envs/ndx-tank-metadata/lib/python3.8/site-packages/hdmf/build/objectmapper.py", line 88, in _ascii raise ValueError("Expected unicode or ascii string, got %s" % type(s)) ValueError: Expected unicode or ascii string, got <class 'datetime.datetime'>

Steps to Reproduce

Snippet to create the extension with a datetime attribute:

import os
from pynwb.spec import NWBNamespaceBuilder, NWBGroupSpec, export_spec

def main():
    ns_builder = NWBNamespaceBuilder(
        doc='extension to demonstrate datetime issue',
        name='datetime-example',
        version='0.1.0',
        author=['xxx'],
        contact=['xxx']
    )

    ns_builder.include_type('LabMetaData', namespace='core')

    LabMetaDataExtension = NWBGroupSpec(
        doc='type for storing metadata',
        neurodata_type_def='LabMetaDataExtension',
        neurodata_type_inc='LabMetaData',
    )

    LabMetaDataExtension.add_attribute(
        name='session_end_time',
        doc='Datetime when session ended',
        dtype='datetime', 
    )

    # export the extension to yaml files in the spec folder
    output_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..', 'spec'))
    export_spec(ns_builder, [LabMetaDataExtension], output_dir)

if __name__ == "__main__":
    main()

Snippet to reproduce the ObjectMapper error message:

import os

from hdmf.build import ObjectMapper
from pynwb import NWBFile, get_class, load_namespaces, NWBHDF5IO, register_map

from datetime import datetime
from dateutil.parser import parse as dateutil_parse

name = 'datetime-example'
spec_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..'))
ns_path = os.path.join(spec_path, 'spec', f'{name}.namespace.yaml')

load_namespaces(ns_path)
LabMetaDataExtension = get_class('LabMetaDataExtension', name)

@register_map(LabMetaDataExtension)
class LabMetaDataExtensionMap(ObjectMapper):

    @ObjectMapper.constructor_arg('session_end_time')
    def dateconversion(self, builder, manager):
        datestr = builder.get('session_end_time').data
        date = dateutil_parse(datestr)
        return date

nwbfile = NWBFile('description', 'id', datetime.now().astimezone())
lab_metadata = LabMetaDataExtension(name='LabMetaData',
                                    session_end_time=datetime.now().astimezone())

nwbfile.add_lab_meta_data(lab_metadata)

filename = 'test_labmetadata.nwb'

with NWBHDF5IO(filename, 'w') as io:
    io.write(nwbfile)

Environment

Python Executable: Conda
Python Version: Python 3.8
Operating System: macOS
HDMF Version: 2.3.0 (pynwb 1.4.0)

cc @bendichter

rly commented 3 years ago

Thanks @weiglszonja . I'll look into this.