Application to public HSC DR3 as proof of concept

[tms-epcc]

From FY24 Q1 QU

[tms-epcc]

08/FEB/23

• uses the independent code from #13 to do this.

[tms-epcc]

07/MAR/24

• with independent code have started looking at this but still to do with TxPipe as per #13

[tms-epcc]

25/MAR/24

• template deprojection applied to HSC and obtained power spectra
• memory issues are blocking systematics so these are being investigated further
• Note this does not affect delivery of D3.8.2.1 PR ie. #1

[tms-epcc]

08/APR/24

• O/G

[tms-epcc]

30/APR/24

• discussed memory issues and possible solns at DESC telcon recently
• PCA looks to be a way to deal with this so will investigate
• also running a prelim analysis at lower resoln for paper
• will then proceed from there

[tms-epcc]

14/MAY/24

• have power spectra for all 3 fields.
• Now need analysis with all relevant systematics deprojected
• Note that this ticket was removed from the D3.8.2.1 Epic (#13) since it was not part of the resulting PR
• this task will however be validating the outputs of D3.8.2.1 (#13) hence the blocking dependency was added

[tms-epcc]

04/JUN/24

• looking at power spectra at low res due to scalability struggles
• have signals where not expecting to be so investigating possible sources eg. could be survey mask so will ask WLSS
• https://docs.google.com/presentation/d/1lWDdQnFzkEQTd0ZqMDAmKaWi9I4osyuD/edit#slide=id.p1
• https://docs.google.com/document/d/1xYt89XlkRJYiJNYvRW5qaOsi7QXY6er4ZLzXTfkE144/edit#heading=h.omuouqr670cc

[tms-epcc]

01/JUL/24

• still looking at the above mentioned signal issues.
• identified possible sources but as yet not been able to mitigate
• sky mask has generated some unwanted cross-correlations
• suspect HSC pipeline is missing an item so this will be looked into
• this will be helpful for DESC regardless

[tms-epcc]

29/JUL/24

• have identified the source of the unwanted cross-correlations
• this is due to cuts in the data at the download stage
• now redoing without the cuts to see what the effect is
• looks likely that this is dealing with the issue so probably no need for simulated data after all

[tms-epcc]

20/AUG/24

• the above approach was successful.
• now have a means to get accurate power spectra
• implementing this as python script
• no need to establish if this needs to incorporate into TxPipe (see Google doc for the masks and the responsibilities -Boris will check)
• next step is to produce first final power spectra

[tms-epcc]

12/SEP/24

• believe have accurate power spectra
• now using these to derive constraints on the distribution model
• also look at improving the linear deprojection method eg. remove assumptions and deal with memory issues

[tms-epcc]

01/OCT/24

• now have code for fitting basic halo distribution model. This gives preliminary result
• Next stage is to improve this values
• 3rd pt above still o/g

[tms-epcc]

21/OCT/24

• halo distributions: rerunning to identify issues

[tms-epcc]

13/NOV/24

• still having issues with the halos, getting poor fits.
• continued testing with HSC DR1
• still investigating these -
• but note the pipeline itself is ok for the purposes of LSST:UK so the final will be to confirm power spectra from the pipelne are realistic from HSC DR1