spedas / pyspedas

Python-based Space Physics Environment Data Analysis Software
https://pyspedas.readthedocs.io/
MIT License
154 stars 60 forks source link

Converting CDF epoch values to datetimes is slow #733

Closed jameswilburlewis closed 3 months ago

jameswilburlewis commented 9 months ago

Xiangning.Chu@lasp.colorado.edu emailed me asking if we could go back to using unix times internally in pytplot, rather than datetimes. They noticed that conversion of CDF epoch values to datetimes is slow -- perhaps 2-3 times slower than using unix times instead. But converting from unix times to datetime64 values is much faster. So they're wondering if we should go back to using unix times internally.

cdf_to_tplot used to work that way, so we need to understand why it was changed to use datetimes internally.

Maybe there's something the cdflib maintainers can do to speed up the conversion?

jameswilburlewis commented 3 months ago

This issue is probably moot, now that PyHC is advocating for a common time representation across projects, which will probably be astropy times.