Updated inter-class divergence scores for BH vs NS vs BH-NS vs PBH #8

[andorsk]

Linked Issue: https://github.com/andorsk/rk_toolkit/issues/8

[andorsk]

Notes here:

Once intra-class is completed/updated with the corresponding loss optimization graph you could then address the inter-class similarity-divergence score between a BH-BH R-K diagram and a NS-NS (such as GW170817) the way you discussed on call. I just need the percentage similarity scores for e.g. GW170817 (NS-NS) is 0.69-0.72 similar to BH-BH diagrams so we can get an average similarity score from there) Once you do one such example for me I shall then try to do the same for GW190814 (PBH-BH) on my own following the same steps as you do the first one.

[andorsk]

Need to review this in more detail. We can mix sets together BH-BH, but finding the difference between classes at a categorical level requires us to have a categorical representation of what a class is.

i.e I can take NS-NS, put them together as a subset, and look at how similar/apart they are. What I can't do is say "this is the average topology of a NS, and this is the average topology of BS, they are X distance from eachother". This is possible, but not today

[animikhroy]

I have made a doc to collect all the results, images and brief descriptions in one place with relevant comments wherever applicable for your consideration when you address this issue today. Here is the link to that results doc for your reference:

IMPORTANT CLARIFICATION TO NOTE

NS-NS is NOT an inter-class score; it is an intra class score and those scores can be moved under the intra class issue. Inter Class would be NS-NS vs BH-BH or BH-BH vs BH-NS or BH-BH vs PBH-BH etc. so this needs to be addressed and rectified today unless you already had it in mind. Even if you cant find the average for now If you can find any reasonable inter-class measure that would be enough to qualify for this criteria.

https://docs.google.com/document/d/13DjS3Y44cE9hJq6H403zvSlkI1o9uMNBhBhK9jaRGQ8/edit?usp=sharing

This google doc is solely focused on the updated results and descriptions for ligo_secondary_analysis.ipynb. I am putting all the issues in sequence and requesting brief descriptions of techniques and methodologies involved with relevant references to cite (if any) based on your work, for me to then elaborate further on those points in details and quickly wrap up the final draft of the paper for peer review.

[andorsk]

I had an idea, which I think gets us there. I took the median values of a class and then generated RK Diagrams. That can represent the "average" RK-Diagram per class. Then we can create distance metric between them.

[animikhroy]

@andorsk I like this idea it sounds perfectly suited to our requirement and makes sense from the math POV. So definitley go ahead with it and remember to update some description on the steps and methods used for this idea on google docs for my reference to expand upon. Thanks!! very cool!

[andorsk]

Fix label:

[andorsk]

@animikhroy please review when you can

[animikhroy]

@andorsk the methodology seems good but the average similarities are still too high for example a high mass black hole is showing 91% similar to a Primordial Black Hole Candidate. Can we maybe knock off all the redundant node clusters except: Mass, Spin, Q & Redshift and redo this process?

Then the similarity scores are bound to be more different and less similar as they are to the naked eye without the additional node clusters apart from the 4 mentioned. Otherwise this doesn't make sense from the actual physics POV.

[andorsk]

@animikhroy if you want, you can submit a new hierarchy file and we can rerun things to see if it improves. Same process as before. It is also possible that maybe the distance measures are not sensitive enough, but we don't have time to address new distance measures now.

[animikhroy]

@andorsk I agree.. let's try the hierarchy did you mean the JSON file? If so could you send it to me once? I shall make a copy and edit the columns and re-share with you asap!

[andorsk]

@animikhroy ... everything is on github. I don't know why you're asking me to send it to you.

[andorsk]

https://github.com/animikhroy/rk_toolkit_pipeline_diagrams/blob/main/02_notebooks/rk_gw_mma/data/gwtc_heirarchy_pretty.json

[animikhroy]

@andorsk since there's been a lot of movement lately and I want to make sure we are referring to the same version not tk waste time.

[animikhroy]

@andorsk got it thanks!

[animikhroy]

@andorsk New JSON hierarchy file sent as a PR! Please check!

[andorsk]

@animikhroy new ontology:

[andorsk]

PBH and NS+Black Hole have the same toplogy.

[animikhroy]

@andorsk this is also a very beautiful scientific validation of our approach. All NS-BH candidates are indistinguishable from PBH candidates using those 4 parameter estimates so we are absolutely on the right track. This actually requires the addition of more parameters like done in the previous step or by introducing electromagnetic counterparts from multi-messenger data. Keep them as they are. The BH high mass and PBH show significant differences which is physically good to explain.

In the paper I shall use both versions of the results and compare. Please keep both hierarchy versions of the result in the secondary analysis notebook for comparison and scientific commentary!

Nothing to worry here! ALL GREAT!

[andorsk]

sweet. Thanks. I'm going to close this issue now and merge!

[animikhroy]

@animikhroy new ontology:

@andorsk these set of results are no longer visible on the ligo_secondary_analysis.ipynb. As discussed with you, I would need both sets of results on the notebook to refer to and compare in the results section of our paper.

[andorsk]

@animikhroy you need to just run the same code but point it to the old ontology file. Where it says "pretty_hierarchy.json" change it to the other json