parsotat / BatAnalysis

A python HEASOFT wrapper for processing Swift-BAT data.
MIT License
28 stars 11 forks source link

add 'BAT_EXPOSURE > 0' filter #12

Closed sfzastro closed 10 months ago

sfzastro commented 10 months ago

The main change is to replace return table with return table[table['BAT_EXPOSURE']>0] to remove those observations with no BAT data from the return of from_heasarc().

There are many extra spaces in the batlib.py file, and these unnecessary spaces are cleared by my editor automatically.

dmopalmer commented 10 months ago

This change would break too many things and probably not do what you want.

The swiftmaster table documentation says of this column:

BAT_Exposure

The BAT exposure in seconds on source. The BAT can run simultaneously several modes. The main modes are the EVENT and SURVEY modes and they are alternated within an observation (never run simultaneously). Within an observation the EVENT mode is used when detecting a burst, otherwise the BAT collect data in the SURVEY mode. The bat_exposure on source is calculated as the sum of the effective exposure on source of the EVENT and SURVEY modes.

This description is either wrong or describes something not useful. BAT is typically in survey mode when it is stably pointing, and it provides event data when requested to by ground command or by on-board processes (e.g. 20 minutes worth surrounding a GRB, 10 s worth when a false trigger occurs). BAT can provide simultaneous event and survey data.

I don't know what the circumstances are where the bat_exposure column reads zero. Spot checking an example

SDSSJ113528.93+00462 | 00038224007 | 11 35 14.89 | +00 45 06.7 | 2023-06-28 22:23:35

The HEASARC archive does not have any BAT data. However, BAT was operating at the time, so it may be an archive problem.

There might be cases where bat_exposure is 0 because there is no BAT survey data, but there might be other data that is useful for other analyses. (e.g. BAT rates data, instrumental values, XRT, UVOT...). Furthermore, the from_heasarc routine is used for other tables, not just swiftmastr, and those other tables will not necessarily have the BAT_EXPOSURE column. Therefore we don't want to filter our data so deep in the code.

Depending on your use case it is probably best to go to a higher level in your code, in order to exclude those observations for which there is no data of the specific type you are using.

Therefore I recommend against this pull request.

(For your comment about extra spaces, I agree and submitted #13 )

sfzastro commented 10 months ago

Besides the excess whitespace and np.int problems (addressed in #13), if the user wants BAT_EXPOSURE > 0, this can be applied as a query parameter instead of a code change.

As an example, to get BAT_EXPOSURE > 0 and, for comparison, BAT_EXPOSURE == 0:

table_batnonzero = ba.from_heasarc(name=None, bat_exposure='>0', start_time="2023-01-01 .. 2023-01-31")
table_batzero = ba.from_heasarc(name=None, bat_exposure=0, start_time="2023-01-01 .. 2023-01-31")
print(f"{len(table_batnonzero)=} {table_batnonzero['BAT_EXPOSURE'].min()=}")
print(f"{len(table_batzero)=} {table_batzero['BAT_EXPOSURE'].max()=}")
len(table_batnonzero)=1000 table_batnonzero['BAT_EXPOSURE'].min()=219.0
len(table_batzero)=18 table_batzero['BAT_EXPOSURE'].max()=0.0

Thanks, I will close this commit.