Closed braingram closed 6 months ago
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 64.95%. Comparing base (
4d7c3a6
) to head (15618ee
). Report is 16 commits behind head on main.:exclamation: Current head 15618ee differs from pull request most recent head ced15e7. Consider uploading reports for the commit ced15e7 to get more accurate results
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
JWST regression tests run with no errors: https://plwishmaster.stsci.edu:8081/blue/organizations/jenkins/RT%2FJWST-Developers-Pull-Requests/detail/JWST-Developers-Pull-Requests/1269/pipeline/199/
Opening a jwst file with many (~550) extensions (level 2 output for nirspec mos mode) take ~9 seconds (when
skip_fits_update=True
). Opening the same file withskip_fits_update=False
takes longer, ~26 seconds. Surprisingly, a large portion of the added time is spent in calls toget_hdu
.Using
cProfile
andskip_fits_update=False
opening now takes ~60 seconds (due to profiling overhead) and the rendered profile (with snakeviz) is as follows:zooming into
_load_from_schema
reveals ~40 seconds inget_hdu
This PR adds a
hdu_cache
to skip repeated indexing the hdulist (which in some conditions for this file takes 2-3 ms). With this PR opening the same file withskip_fits_update=False
(and no profiling) takes 12 seconds and withskip_fits_update=True
still takes 9 seconds (most of this is spent inasdf.open
as the tree is quite large). RunningcProfile
withskip_fits_update=False
takes 24 seconds and zooming in toget_hdu
reveals 5 seconds spent inget_hdu
(20% down from 66% without this PR):Checklist
CHANGES.rst
(either inBug Fixes
orChanges to API
)