ranaroussi / pystore

Fast data store for Pandas time-series data
Apache License 2.0
562 stars 101 forks source link

Append doesn't work on datetimes #9

Closed Frankkkkk closed 5 years ago

Frankkkkk commented 5 years ago

Hi, First of all, thank you for this project ;-)

I've got a little problem with the append function: it doesn't do anything with the indexes are datetimes:

pystore.set_path('/dev/shm/test/pystore')
store = pystore.store('mydatastore')
collection = store.collection('collection')

item = 'test'

it1 = pd.DataFrame({'data': {datetime.datetime(2019, 1, 1): random.random()}})
collection.write(item, it1, overwrite=True)

it2 = pd.DataFrame({'data': {datetime.datetime(2019, 1, 2): random.random()}})
collection.append(item, it2)

it = collection.item(item)
df = it.to_pandas()
print(df)

yields:

                data
index               
2019-01-01  0.450675

However, if I use a timestamp instead of a datetime (datetime.datetime(xxx).timestamp()), it works:

                         data
index                        
2018-12-31 23:00:00  0.385200
2019-01-01 23:00:00  0.639432

Thanks

Frankkkkk commented 5 years ago

FYI: the problem may be that data.index.dtype_str is datetime64[ns] instead of datetime

ranaroussi commented 5 years ago

Can't reproduce. Closing this for now...