search

DavidT3 / XGA

X-ray: Generate and Analyse is a module designed to make the analysis of XMM observations simple and efficient. It provides an interface with SAS for the creation of XMM data products, as well as a way to easily perform fits (scalable for multiple observations) and retrieve information about an object, all within a Python package.
BSD 3-Clause "New" or "Revised" License
29 stars 3 forks source link

Duplicate events in different eRASS Obs_IDS #1151

Open jessicapilling opened 5 months ago

jessicapilling commented 5 months ago

The way eRASS-1 has been released is a bit problematic for XGA.

They have divided the sky into 'skytiles' which is the equivalent to an obs_id.

Each skytile has a unique 3.6 x 3.6 area. Surrounding this area is a variable area where all the events are duplicated in other skytiles/ obs_ids.

This is not good for XGA since we take all event lists from all associated observations to do analysis, so we are currently counting these duplicate events as different events. So this means we will over estimate the counts in the clusters if they fall in the overlap region.

Also XGA only makes images of the 3.6 x 3.6 region of an obsid. This means that if a cluster falls on an overlap region, it will not appear in XGA generated images and therefore exposure maps as well.

We need to find a sensible way to deal with this so that the user can see the individual observations that the cluster appears in and make X-ray products from all associated obs, but have XGA ignore duplicate events when doing any anaylsis of the products.

jessicapilling commented 5 months ago

A quick fix is to limit the search_distance argument for erosita, and only allow a search_distance of 1.8deg, this will only collect sources within the unique area of the skytile, and not detect any in the duplicated area.

jessicapilling commented 5 months ago

I made some diagrams to help articulated the problem:

Screen Shot 2024-05-30 at 11 15 57 Screen Shot 2024-05-30 at 11 16 03 Screen Shot 2024-05-30 at 11 16 13 Screen Shot 2024-05-30 at 11 16 21 Screen Shot 2024-05-30 at 11 16 30 Screen Shot 2024-05-30 at 11 16 38 Screen Shot 2024-05-30 at 11 16 46

Obviously some cases are more rare than others, but ideally XGA will be able to cope with all of them

jessicapilling commented 5 months ago

The trouble with limiting the search radius is that we will completely miss sources in case 2 and 3.

We will also still over-count events for sources in cases 5 and 6.

jessicapilling commented 5 months ago

I think to deal with the problem properly:

  1. Upon declaration see how many sources are associated
  2. Have a property called ._analysis_obs
  3. In cases 1,2,3, and 4, can select one observation
  4. In cases 5, and 6, use evtool to merge the observations, assign this merged event list to the analysis obs property
  5. In sourcetools / where analysis happens, have a line: if telescope = erosita, obs_id = self._analysis_obs
jessicapilling commented 5 months ago

New plan:

  1. Write a combine event lists generating function --> esass.generate.
    • these are stored in the 'combined' level in the product storage heirarchy
  2. evtool_image, expmap --> optional argument combined_obs
  3. srctool_spectrum --> combine_tm = True, --> combine_obs --> True, raises warning if set to False
  4. fitting/ analysis stuff tbc.
jessicapilling commented 5 months ago

Make sure to consider erositacalpv

jessicapilling commented 4 months ago

Step 1: TODO

  1. add a table reading function into the Eventlist class
  2. see if you can only read certain columns into memory - if this isnt in astropy, could be in fitsio, read_header_on_demand and unload function exist in XGA - already in EventList class
  3. could have a get_specific_data method, specify which columns to use
jessicapilling commented 4 months ago

Step 2: TODO

  1. What should the default size for merged images be?
  2. combined images and exposure maps need to have a randint in the name