Healpix prior #477

Closed bilby-bot closed 1 month ago

In GitLab by @git.ligo:gregory.ashton on Oct 7, 2019, 22:40

Add a method to read in and use a healpix prior (e.g. from Fermi/INTEGRAL). I suspect this will need some close interaction with the correlated prior.

In GitLab by @git.ligo:jonathan.merritt on Oct 8, 2019, 20:28

I would like to take this on.

In GitLab by @git.ligo:zoheyr-doctor on Oct 8, 2019, 20:28

unassigned @git.ligo:zoheyr-doctor

In GitLab by @git.ligo:zoheyr-doctor on Oct 8, 2019, 20:31

I've added myself as the assignee for now until @git.ligo:jonathan.merritt is added to the project.

In GitLab by @git.ligo:gregory.ashton on Oct 8, 2019, 23:17

Great, I've added @git.ligo:jonathan.merritt to the project and also assigned him.

A word of caution on this one: this is a special case of a correlated prior (between ra/dec and potentially distance), the implementation of which !332 is currently still in progress (cc @git.ligo:sylvia.biscoveanu, @git.ligo:moritz.huebner, and @git.ligo:matthew-pitkin who have all variously worked on this). I'm not sure how it is best implemented, but there is lots of potential conflict so I suggest everyone make sure to communicate to avoid code clashes :)

In GitLab by @git.ligo:gregory.ashton on Oct 8, 2019, 23:18

That being said: the reading on of healpix priors by itself might be quite a task so that is where I'd personally start.

In GitLab by @git.ligo:zoheyr-doctor on Oct 9, 2019, 01:29

Thanks, @git.ligo:gregory.ashton, that's good to know. Just trying to catch up on what's been done on the correlated prior issue, and if I'm understanding correctly, @git.ligo:jonathan.merritt and I would need to plug a healpix map into the correlated prior infrastructure (which I will have to familiarize myself with)?

In GitLab by @git.ligo:jonathan.merritt on Oct 9, 2019, 18:05

@git.ligo:moritz.huebner @git.ligo:sylvia.biscoveanu @git.ligo:matthew-pitkin We're looking at the conditional prior issue on branch 270: https://git.ligo.org/lscsoft/bilby/blob/270-introduce-correlated-prior-sets/bilby/core/prior.py Can the prior sampler only check if the samples in the conditional variable satisfy a constraint of the unconditional variable or is there an existing way to feed in a joint probability distribution or p(conditional var|unconditional var)? Thanks.

In GitLab by @git.ligo:zoheyr-doctor on Oct 9, 2019, 18:26

For the LIGO public-facing skymaps for example, we get a healpix map with columns p(pixel) and p(distance|pixel). Is there a natural way this fits into the conditional prior infrastructure? Perusing the prior.py code made me think that you can't quite do this?

In GitLab by @git.ligo:moritz.huebner on Oct 11, 2019, 03:10

@git.ligo:jonathan.merritt @git.ligo:zoheyr-doctor sorry for the late response. I just came back to Melbourne from Europe. I think p(conditional var|unconditional var) is pretty much what I am implementing right now, so we could definitely handle something like p(distance|pixel).

The only major issue right now is there is no way to save/load these priors to/from files, because the condition is implemented via a function which you can't easily save. I don't think I will implement this feature in the current MR.

I'll present the progress on this MR on the bilby dev call on Monday. I am only adding tests right now and fixing some technical issues that I come across.

In GitLab by @git.ligo:zoheyr-doctor on Oct 11, 2019, 18:42

No worries, @git.ligo:moritz.huebner. Perhaps it makes most sense to discuss this on the call on Monday.

In GitLab by @git.ligo:zoheyr-doctor on Oct 15, 2019, 01:26

Thanks for presenting on the dev call @git.ligo:moritz.huebner. Now I have a sense of what you've implemented. I imagine what would happen would be something like:

my_healpix_file = get_my_healpix_file()
def conditional_func_distance(reference_params,ra,dec):
    return dict(mu=my_healpix_file.distmu(ra,dec),
                sigma=my_healpix_file.distsigma(ra,dec)
               )

conditional_prior_dict['distance']=bilby.core.prior.Gaussian(
                                   condition_func = conditional_func_distance, ...
                                   )

Alternatively, we could skip over using the conditional prior stuff you've built and instead implement a whole new Healpix prior object class. It would be initialized with healpix file and then have sample, prob, ln_prob, and cdf methods.

In GitLab by @git.ligo:moritz.huebner on Oct 15, 2019, 03:52

@git.ligo:zoheyr-doctor I was thinking something like that. I am not too familiar with the nature of the Healpix priors, but if they do indeed follow analytical distributions, it should be as easy as that. But even if we have just a number of points with associated probability, we could manage to do that with Interped (interpolated) priors.

I would nicely put this desired behaviour away into a separate function, so that you only have to write something like:

my_healpix_file = get_my_healpix_file()
priors = ConditionalPriorDict()
priors['ra'], priors['dec'], priors['distance'] = get_prior_from_healpix(my_healpix_file)

I also do apologise for the poor audio earlier.

In GitLab by @git.ligo:cjhaster on Oct 21, 2019, 18:38

this is partially a followup of the features I presented in https://git.ligo.org/lscsoft/bilby/issues/293.

I think the HealPIX implementation is the way forward (note that we need to support both 2D and 3D FITS files), but that it could also be useful to have a more simple Gaussian skyBlob implemented, but I think that might be covered more by https://git.ligo.org/lscsoft/bilby/merge_requests/332?

In GitLab by @git.ligo:bruce.edelman on Nov 13, 2019, 00:22

Hi all, We have been working on this, going with the approach of creating a general JointPrior class for similar types of priors in the future. We started by taking @git.ligo:matthew-pitkin 's MultivariateGaussian prior infrastructure and try to generalize it so that all we have to do to create joint priors is to create a Prior that inherits from BaseJointPriorDist then for each parameter included create a JointPrior that stores the instance of the JointPriorDist that it is a part of. For more details see my branch here:

https://git.ligo.org/bruce.edelman/bilby/blob/joint_prior/bilby/core/prior.py#L2437 https://git.ligo.org/bruce.edelman/bilby/blob/joint_prior/bilby/gw/prior.py#L635

We have only implemented 2D HealPix priors for now but it should be straightforward to extend to 3D (if what is here is correct and working). We were hoping to get some insight from other bilby developers here to see if this approach is satisfactory with everyone in terms of infrastructure to make sure there isn't any large conflicts of infrastructure we aren't thinking about.

Curious to here all your thoughts, Thanks!

In GitLab by @git.ligo:gregory.ashton on Feb 6, 2020, 03:39

closed via merge request !651

In GitLab by @git.ligo:gregory.ashton on Feb 6, 2020, 03:39

mentioned in commit 38d1effce5cf2e4e2ab2336f7e687a9251475712

In GitLab by @git.ligo:michael.williams on Oct 3, 2024, 17:55

unassigned @git.ligo:zoheyr-doctor, @git.ligo:bruce.edelman, and @git.ligo:jonathan.merritt

bilby-dev / bilby

Healpix prior #477