Tapering behavior for Observation is confusing

[pllim]

@mgennaro is going to post more details about this. In a nutshell, when bandpass data ends abruptly without being tapered, Observation extrapolates the bandpass and there does not seem to be an easy way to control that behavior. On the contrary, when source spectrum data ends abruptly, it can be controlled using force keyword.

[mgennaro]

Thanks @pllim for filing this...and sorry for my delay. I don't have much to add to your description except:

• The problem can be circumvented by explicitly creating an observation and passing a tapered bandpass.

obs  = sp.Observation(spectrum, b.taper())

• At the same time I believe this behavior is not desirable. While it may make sense in certain cases to extrapolate a spectrum, I cannot think of a case where it would be beneficial to extrapolate a bandpass beyond the point where a bandpass is defined, especially if the last bandpass valid value (the one that gets extrapolated as a constant) is not-zero. I think the default behavior should be for the bandpass to be tapered, leaving the option of a different behavior to interested users.

• this is possibly relevant to JWST zeropoint calculation as well as ETC calculations. tagging @bhilbert4 @ariedel

[pllim]

I vaguely remember a campaign some time ago to explicitly taper all the HST throughput files to avoid just this problem, or am I just imagining it? Maybe @rizeladiaz or @vglaidler remember.

[pllim]

For the record, I am not opposed to changing the behavior but there is backward compatibility issue to consider, as this effects ETC itself. Maybe I should ping @ibusko too.

[ibusko]

The HST ETC uses force='taper' whenever a PartialOverlap exception is raised by Pysynphot (not the new synphot! Since the code wasn't refactored yet to connect to synphot).

[vglaidler]

vaguely remember a campaign some time ago to explicitly taper all the HST throughput files to avoid just this problem, or am I just imagining it? Maybe @rizeladiaz or @vglaidler remember.

Yes, this is correct. The rationale is that the software should not be trying to guess what to do; the user should provide data that has the correct behavior. That is the most transparent and explicit approach, and thus safest from the perspective of "is the ETC giving the correct answer".

In the original pysynphot, the force keyword was designed to handle the situation of partial overlap either by tapering to zero, or by extrapolating (don't recall whether it was linear extrapolation, or extrapolation at constant value).

Note that this design decision was reached by a combined group of software folks and scientists, trying to handle all use cases both correctly and with, at least, an ease-of-use option for users.

A significant topic in this discussion was that "absence of data does not equal zero data": that is, if a throughput file (or spectrum file, for that matter) contains no data, it is explicitly not correct to fill the missing data with zeros.

[pllim]

I think the conclusion here is that we won't change the software behavior.