ODM2 / ODM2PythonAPI

A set of Python functions that provides data read/write access to an ODM2 database by leveraging SQLAlchemy.
http://odm2.github.io/ODM2PythonAPI/
BSD 3-Clause "New" or "Revised" License
4 stars 13 forks source link

SQLite Result ValueDateTime regression to non-datetime type #158

Closed emiliom closed 5 years ago

emiliom commented 5 years ago

getResultValues on a measurement-type SQLite ODM2 database is now returning ValueDateTime types as strings rather than datetime.

This was not the case last November, when the same code with the same database returned datetime types. In this notebook from the BiGCZ workshop in early-mid November 2017, the SQLite database is examined extensively. The time series plots in cells 24 and 26 correctly show valuedatetime as a datetime. Running that code today with a conda environment created back then, with odm2api 0.6.0a (created May 2017), also produces the expected type. But running it with the current odm2api release (0.7.1) returns a unicode-string-type valuedatetime.

I've traced back changes that could have led to this regression (and bug). I think I've found it in this commit from 2017-11-13 ("update tests, update DateTime data type to be compatible with sqlite", merged in PR #122), in its changes to models.py. Specifically: https://github.com/ODM2/ODM2PythonAPI/commit/8a8ffe7d795cab657798c7149c9d4355e5c47bb5#diff-0ee3d0270e8061c9eb0e712c2e83d27aR18 and https://github.com/ODM2/ODM2PythonAPI/commit/8a8ffe7d795cab657798c7149c9d4355e5c47bb5#diff-0ee3d0270e8061c9eb0e712c2e83d27aR1658 (as well as all other ResultValues classes)

I'm hoping the fix will be simple, involving changing https://github.com/ODM2/ODM2PythonAPI/blob/master/odm2api/models.py#L17 from

DateTimeType = DateTimeType.with_variant(sqlite.INTEGER(), 'sqlite')

to

DateTimeType = DateTimeType.with_variant(sqlite.DATETIME(), 'sqlite')

But I haven't tested it yet.

emiliom commented 5 years ago

Fixed by #169. Closing