search

ranaroussi / pystore

Fast data store for Pandas time-series data
Apache License 2.0
562 stars 101 forks source link

Integer index is converted to datetime automatically #10

Closed BlackArbsCEO closed 5 years ago

BlackArbsCEO commented 5 years ago

Hi I'm experimenting with pystore and just realized that an integer index is converted automatically to epoch datetime when reading the stored data:

tmp_df.head()
Out[69]: 
      askPrice  askSize      ...        mid_price  dollar_volume
2524   65.9800      100      ...          65.9550     6,596.0000
3155   65.9800      100      ...          65.9550     6,596.0000
3786   66.0100     2500      ...          65.9400     6,596.0000
4417   66.0200     2500      ...          65.9800     6,596.0000
5048   66.0200     2500      ...          65.9800     6,596.0000
[5 rows x 19 columns]
collection.append(symbols[0], tmp_df.reset_index(drop=True))
df1 = collection.item(symbols[0]).to_pandas()
cprint(df1)
-------------------------------------------------------------------------------
dataframe information
-------------------------------------------------------------------------------
                               askPrice       ...         dollar_volume
index                                         ...                      
1970-01-01 00:00:00.000325455   67.2100       ...       16,339,954.8800
1970-01-01 00:00:00.000325456   67.2100       ...       16,339,954.8800
1970-01-01 00:00:00.000325457   67.2100       ...       16,810,389.8800
1970-01-01 00:00:00.000325458   67.2500       ...       16,962,429.2400
1970-01-01 00:00:00.000325459   67.2500       ...       16,962,429.2400

I'm not sure if this is expected behavior as it wasn't stated in the examples that pystore requires a datetime index.

ranaroussi commented 5 years ago

By default, PyStore tries to convert indices to datetime. You can override this directive by using item.to_pandas(parse_dates=False).