Closed JLBLine closed 1 month ago
The part above that says _"Now, Component_TypeCounter is setup to still be counting things iteratively, as if it was reading things in from a yaml or text file. I could take this opportunity to strip out this code and stream line." is wrong. I still use those text file style things in read_yaml_radec_count_components
, so leave that code in there for now
In ddf2b32ec903a0a4b395096534fe6b2fa9619c42 I have modified read_fits_radec_count_components
to count and index polarisation info from the FITS table
In 2cb43ec059aa9078ab5a82a6500f855bf13461f9 enabled create_skymodel_chunk_map
to handle polarisation information. Tested by updated and adding to tests in cmake_testing/wodenpy/skymodel
OK, read_skymodel_chunk
can read polarisation info from a FITS file as of c8688f8f30b01c75ba08e8548b6f4f457e3d35fd. Tests have been embellished/updated in cmake_testing/wodenpy/skymodel
Sigh. Unless stated, all references to Python code in this section are in
wodenpy.skymodel
. Makes things easier to read.The major order of sky model things that happens in
run_woden.py
is as follows:read_skymodel.read_radec_count_components
to count how many components of each type we have in the sky model, as well as the RA/Dec positions, by creating and filling awoden_skymodel.Component_Type_Counter
objectwoden_skymodel.crop_below_horizon
to crop out stuff below the horizon; results are fed into theComponent_Type_Counter
Component_Type_Counter
intochunk_sky_model.create_skymodel_chunk_map
, which returns a list ofComponents_Map
objects. Each of these maps contains all the indexes of components from the sky model to make a chunk that will fit into GPU memoryComponents_Map
objects is then fed into therun_woden.read_skymodel_thread
thread, which reads in a number of chunks of the sky model, sends it to the GPU (which will iterate over however many chunks were sent across), and then reads the next set of chunks while the GPU is running the last set.run_woden.read_skymodel_thread
callsread_skymodel.read_skymodel_chunks
to read in the sky model chunks. This in turn callsread_fits_skymodel.read_fits_skymodel_chunks
, which sets up awodenpy.use_libwoden.skymodel_struct.Source_Catalogue_Double/Float
class. This is filled with a "Source" for every "chunk", and all the "components" associated with that chunk are populated into that "Source". ThisSource_Catalogue_Double/Float
is actypes
object that is passed directly to the GPU code.The difficult part here is the "chunking" of the sky model, which not only splits the sky model up into smaller chunks for the GPU to handle, but also reorders by POINT/GAUSSIAN/SHAPELET, as well as by Stokes I power-law/curved power-law/list-type. We'll need to setup ways to track linear and circular polarisation along with the power-law/curved power-law/list-type. That means editing the things listed below.
At the end result that goes into the GPU code, we have the
wodenpy.use_libwoden.skymodel_struct.Components_Double/Float
class, which are attributes of thewodenpy.use_libwoden.skymodel_struct.Source_Double/Float
class (three of them, one each for POINT, GAUSSIAN, and SHAPELET components).Components_Double/Float
class containspower_comp_inds
curve_comp_inds
list_comp_inds
arrays, which index the flux model info to theextrap_stokesI
(and RA/DEC/Beam) information. I think we need to add the following arrays to theComponents_Double/Float
class:Components_Double.rm_values
- all the rotation measure valuesComponents_Double.linpol_frac
- all the linear polarisation fraction valuesComponents_Double.intr_pol_angle
- all intrinsic polarisation angles; these should be zero if not providedComponents_Double.linpol_comp_inds
- indexes the linear pol info to the overall component index. This will allow us to index theextrap_stokesI
array, which will be calculated from Stokes I info, and we need for calculating linear polarisation.Components_Double.linpol_zero_I_comp_inds
- if user wants to set Stokes I to zero after calculating Q/U; indexesextrap_stokesI
Components_Double.Vpol_frac
- all the linear polarisation fraction valuesComponents_Double.Vpol_frac_comp_inds
- indexes the linear pol info to the overall component index. This will allow us to index theextrap_stokesI
Components_Double.V_power_SIs
- if user wants bespoke Stokes V power-law, need the SIComponents_Double.V_curve_SIs
- if user wants bespoke Stokes V curved power-law, need the SIComponents_Double.V_curve_qs
- if user wants bespoke Stokes V curved power-law, need the q paramComponents_Double.V_power_comp_inds
- if user wants bespoke Stokes V power-law, need to indexextrap_stokesI
Components_Double.V_curve_comp_inds
- if user wants bespoke Stokes V curved power-law, need to indexextrap_stokesI
Maybe rename the assumed Stokes I arrays to be clear:
Components_Double.power_SIs
toI_power_SIs
Components_Double.curve_SIs
toI_curve_SIs
Components_Double.curve_qs
toI_curve_qs
Components_Double.power_comp_inds
toI_power_comp_inds
Components_Double.curve_comp_inds
toI_curve_comp_inds
At the same time, we should delete:
Components_Double.power_ref_stokesQ
Components_Double.power_ref_stokesU
Components_Double.curve_ref_stokesQ
Components_Double.curve_ref_stokesU
as these are wrong and should never have existed. I imagine we'll need to add something like
Source_Double.n_linpol
Source_Double.n_V_polfracs
Source_Double.n_V_point_powers
Source_Double.n_V_point_powers
Source_Double.n_V_point_curves
Source_Double.n_V_gauss_powers
Source_Double.n_V_gauss_curves
Source_Double.n_V_shape_powers
Source_Double.n_V_shape_curves
as we'll need those to tell the GPU which kernels to launch.
To be able to populate these new
Components_Double
arrays, we need to edit the following functions/classes, for each step listed above.read_fits_sky_model.read_fits_radec_count_components
counts how much of everything we have in the sky model. Bear minimum this needs to check for how many linear and/or circular polarisation components we have. We'll need to add some attributes to thewodenpy_skymodel.Component_Type_Counter
class to track these. Now,Component_Type_Counter
is setup to still be counting things iteratively, as if it was reading things in from ayaml
or text file. I could take this opportunity to strip out this code and stream line, as I convertyaml
intoFITS
internally at the moment (which is hella inefficient, should maybe change that also).Will need to add something like the below, as we have to separate out the POINT,GAUSSIAN,SHAPELET types:
Component_Type_Counter.linpol_point_inds
- track Linear pol componentsComponent_Type_Counter.linpol_gauss_inds
- track Linear pol componentsComponent_Type_Counter.linpol_shape_inds
- track Linear pol componentsComponent_Type_Counter.V_point_polfrac_inds
- track Stokes V polarisation frac stuffComponent_Type_Counter.V_gauss_polfrac_inds
- track Stokes V polarisation frac stuffComponent_Type_Counter.V_shape_polfrac_inds
- track Stokes V polarisation frac stuffComponent_Type_Counter.V_point_power_inds
- track Stokes V power/curved power law stuffComponent_Type_Counter.V_point_curve_inds
- track Stokes V power/curved power law stuffComponent_Type_Counter.V_gauss_power_inds
- track Stokes V power/curved power law stuffComponent_Type_Counter.V_gauss_curve_inds
- track Stokes V power/curved power law stuffComponent_Type_Counter.V_shape_power_inds
- track Stokes V power/curved power law stuffComponent_Type_Counter.V_shape_curve_inds
- track Stokes V power/curved power law stuffand a number of associated edits to get
read_fits_radec_count_components
to populate these new arrays. Once again, we should probably rename the assumed Stokes I arrays to be clear:Component_Type_Counter.point_power_inds
toI_point_power_inds
Component_Type_Counter.point_curve_inds
toI_point_curve_inds
Component_Type_Counter.gauss_power_inds
toI_gauss_power_inds
Component_Type_Counter.gauss_curve_inds
toI_gauss_curve_inds
Component_Type_Counter.shape_power_inds
toI_shape_power_inds
Component_Type_Counter.shape_curve_inds
toI_shape_curve_inds
The when we pass that to
create_skymodel_chunk_map
, we need to add the new arrays to theComponents_Map
class, which is populated from theComponent_Type_Counter
class. We'll need something like:Components_Map.linpol_orig_inds
Components_Map.V_polfrac_orig_inds
Components_Map.V_power_orig_inds
Components_Map.V_curve_orig_inds
Components_Map.V_power_shape_basis_inds
Components_Map.V_curve_shape_basis_inds
and should probably rename the assumed Stokes I arrays to be clear:
Components_Map.power_orig_inds
toI_power_orig_inds
Components_Map.curve_orig_inds
toI_curve_orig_inds
Components_Map.power_shape_basis_inds
toI_power_shape_basis_inds
Components_Map.curve_shape_basis_inds
toI_curve_shape_basis_inds
Right, so
create_skymodel_chunk_map
callsmap_chunk_pointgauss
andmap_chunk_shapelets
. The former calls in turn callincrement_flux_type_counters
, which does some insane logic to split things by flux type.map_chunk_shapelets
is different, as it splits things byshapelet coefficient
rather than flux type.This is where it gets tricky. Every polarisation component is associated with some Stokes I component, so we don't need to add any extra logic to split things by polarisation model type. What we do need is some logic in
chunk_sky_model.fill_chunk_component
, that matches any polarisation information we have with the relevant component. Gonna need some serious logic and tested there, with many new variables and bits ofnp.where
methinks. But this is where we populate the new arrays in theComponents_Map
class.Once that crucial bit of mapping logic is correct, we then need to edit
read_fits_skymodel.add_fits_info_to_source_catalogue
to use the new arrays in theComponents_Map
to correctly populate the new arrays inComponents_Double/Float
, the things that get passed to the GPU code.Testing need to update
cmake_testing/wodenpy/skymodel/test_read_FITS_skymodel_chunk.py
to check all these news values are read in, and updatecmake_testing/wodenpy/skymodel/common_skymodel_test.py
andcmake_testing/wodenpy/skymodel/read_skymodel_common.py
with new polarisation options. Should probably write a new testtest_read_yaml_radec_count_components.py
as we don't explicitly test that at the moment (it's tested withintest_read_FITS_skymodel_chunk.py
). Need to make sure a variety of different