sot / mica

Microscope on Chandra aspect
https://sot.github.io/mica
BSD 3-Clause "New" or "Revised" License
3 stars 0 forks source link

Improve speed of getting local OCAT data #272

Closed taldcroft closed 2 years ago

taldcroft commented 2 years ago

Description

Improve the speed of reading the local OCAT. Most of the time is spent decoding the UTF-8

Interface impacts

None.

Testing

Unit tests

Independent check of unit tests by Javier

Functional tests

Current release

(ska3) ➜  docs git:(ocat-local-faster) ipython
Python 3.8.12 (default, Oct 12 2021, 06:23:56) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.29.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from mica.archive.cda import get_ocat_local
In [2]: %time dat = get_ocat_local()
CPU times: user 1.69 s, sys: 83 ms, total: 1.77 s
Wall time: 1.77 s

New using current compressed HDF5

(ska3) ➜  mica git:(ocat-local-faster) ipython
Python 3.8.12 (default, Oct 12 2021, 06:23:56) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.29.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from mica.archive.cda import get_ocat_local
In [2]: %time dat = get_ocat_local()
CPU times: user 456 ms, sys: 93.3 ms, total: 549 ms
Wall time: 549 ms

New using uncompressed HDF5

In [2]: %time dat = get_ocat_local(datafile="ocat.h5")
CPU times: user 298 ms, sys: 103 ms, total: 401 ms
Wall time: 403 ms
taldcroft commented 2 years ago

I fixed the thing about using an observer name, though that comments seems to be gone now.