SpatioTemporal / STAREPandas

STAREpandas adds SpatioTemporal Adaptive Resolution Encoding (STARE) support to pandas DataFrames. https://starepandas.readthedocs.io/en/latest/
MIT License
4 stars 1 forks source link

enforced little-endianess #145

Open mbauer288 opened 1 year ago

mbauer288 commented 1 year ago

To my understanding the numpy dtype <M8[ns] is the byte-order equivalent to datetime64[ns] on little-endian hardware, whereas on big-endian machines datetime64[ns] == >M8[ns].

So we are enforcing little-endianess of our dataframes in tivs_from_timeseries()?

    class STAREDataFrame(geopandas.GeoDataFrame):
        make_tids():
            if not pandas.api.types.is_datetime64_any_dtype(start_col.dtype): <= Allowing any datetime64 object
                raise TypeError('dtype of column must be numpy.datetime64')

            # which calls
            temporal_conversions.tivs_from_timeseries():
                if not series.dtype == '<M8[ns]':         <= enforcing little-endianess?

Not that I can even think of a big-endian machine nowadays, but perhaps we should add a note (or issue a warning) to this?

import sys
sys_is_le = sys.byteorder == 'little'

I say this because by default panadas and datetime make datetime64[ns] objects which defaults to that machine's byte-order.

Also, am I correct that STATEPandas does not want/support datatimes with timezone (UTC) encoding (i.e., not '<M8[ns, UTC]', 'datetime64[ns UTC]')?

NiklasPhabian commented 1 year ago

really good question. I'd assume we can safely remove the little-endianness enforcement. It should not make a difference. I think we really just wanted to test if those are nanoseconds; not the byte-order. Most lazy approach to clear this up would be adding tests.

Also, am I correct that STATEPandas does not want/support datatimes with timezone (UTC) encoding (i.e., not '<M8[ns, UTC]', 'datetime64[ns UTC]')?

Just not yet! But we could easily tweak that. I.e. just enforcing conversion to datetime64[ns UTC].