I've been using the new C-based ramp fitting code pretty extensively lately, and the one oddity that I've noticed was from processing jw02079004003_03101_00001_nis_uncal.fits
The C code runs fine, and fast (23 seconds) but then ramp fitting spends an additional 2-3 minutes before exiting from the ramp fitting routine. I don't recall seeing a delay this lengthy before, it's usually a second or two.
Digging into a profile, it looks like the hangup is in the compute_RN_variances step (i.e., nothing to do with the C routine, but this is the next tall pole, accounting for 82% of total runtime).
Here's the time stamps:
2024-06-25 10:37:18,531 - stpipe.RampFitStep - INFO - Using READNOISE reference file: /Users/dlaw/crds_cache/jwst_ops/references/jwst/niriss/jwst_niriss_readnoise_0005.fits
2024-06-25 10:37:18,531 - stpipe.RampFitStep - INFO - Using GAIN reference file: /Users/dlaw/crds_cache/jwst_ops/references/jwst/niriss/jwst_niriss_gain_0006.fits
2024-06-25 10:37:18,572 - stpipe.RampFitStep - INFO - Using algorithm = OLS_C
2024-06-25 10:37:18,572 - stpipe.RampFitStep - INFO - Using weighting = optimal
2024-06-25 10:37:42,425 - stpipe.RampFitStep - INFO - Ramp Fitting C Time: 22.279687881469727
2024-06-25 10:40:05,451 - stpipe.RampFitStep - INFO - Results used CRDS context: jwst_1241.pmap
2024-06-25 10:40:05,453 - stpipe.RampFitStep - INFO - Results used CRDS context: jwst_1241.pmap
2024-06-25 10:40:05,553 - stpipe.RampFitStep - INFO - Saved model in jw02079004003_03101_00001_nis_0_rampfitstep.fits
2024-06-25 10:40:05,815 - stpipe.RampFitStep - INFO - Saved model in jw02079004003_03101_00001_nis_1_rampfitstep.fits
2024-06-25 10:40:05,815 - stpipe.RampFitStep - INFO - Step RampFitStep done
I'm not sure why this NIRISS example is so different that the other cases I've seen.
Testing on jw01617001001_02101_00001_nrcblong_uncal.fits (8 frames/group, 5 groups/int, 10 ints) runtime is 1 second, but the NIRISS example (4 frames/group, 20 groups/int, 6 ints) is 2.5 minutes.
Issue JP-3669 was created on JIRA by David Law:
I've been using the new C-based ramp fitting code pretty extensively lately, and the one oddity that I've noticed was from processing jw02079004003_03101_00001_nis_uncal.fits
The C code runs fine, and fast (23 seconds) but then ramp fitting spends an additional 2-3 minutes before exiting from the ramp fitting routine. I don't recall seeing a delay this lengthy before, it's usually a second or two.
Digging into a profile, it looks like the hangup is in the compute_RN_variances step (i.e., nothing to do with the C routine, but this is the next tall pole, accounting for 82% of total runtime).
Here's the time stamps:
I'm not sure why this NIRISS example is so different that the other cases I've seen.
Testing on jw01617001001_02101_00001_nrcblong_uncal.fits (8 frames/group, 5 groups/int, 10 ints) runtime is 1 second, but the NIRISS example (4 frames/group, 20 groups/int, 6 ints) is 2.5 minutes.