search

MAVENSDC / cdflib

A python module for reading NASA's Common Data Format (cdf) files
MIT License
82 stars 45 forks source link

Astropy epoch performance generally bad #65

Open bryan-harter opened 5 years ago

bryan-harter commented 5 years ago

While each epoch class uses the same functions, Astropy can be much slower. It is faster for getting the unixtime and encode functions, but it is MUCH slower for the compute and parse functions.

Generally this seems to be stemming from the time it takes to convert an astropy time object to the cdf format, so for example:

Time(1000, format='unix').cdf_tt2000

takes up a lot of time.

We should figure out where the slowdown is happening in astropy, and determine if there is something we can do to speed it up. If not, we should just default to utilizing the functions in the primary CDFepoch module.

bryan-harter commented 5 years ago

This is a bit of code I was using to benchmark the times.

scratch.txt

Cadair commented 5 years ago

If you can pin point any bottlenecks I am sure astropy would be very happy to help out with removing them :smile:

amanotk commented 4 years ago

Is this the reason why cdflib/__init__.py imports CDFAstropy as cdfastropy ? This makes unixtime() much slower again in pyspedas.

amanotk commented 4 years ago

I now found the problem is actually on pyspedas side. cdf_to_tplot used in pyspedas/mms is somehow not the one in pytplot.