LSSTDESC / rail_base

Base classes for RAIL
MIT License
0 stars 1 forks source link

Implent CC summarizer "yet_another_wizz" #17

Open jlvdb opened 2 years ago

jlvdb commented 2 years ago

Implement the original code at https://github.com/jlvdb/yet_another_wizz to fit into RAIL and improve the performance to match the scale of the future datasets.

Main challenges:

aimalz commented 2 years ago

Given that RAIL now speaks ceci, it might be worth looking into what existing infrastructure from TXPipe could potentially be called in a RAIL stage specifically for the cross-correlation step. @joezuntz has thought about this a bunch and probably has additional ideas for efficiency.

sschmidt23 commented 2 years ago

If I remember correctly that YAW is similar to the-wizz in switching from actual correlation function calculations to KDTree matching, then we probably don't need any of the actual TreeCorr stuff. But, I think we do need to figure out how to do proper mask generation/use and generating randoms in a way that is consistent with how other DESC people are doing those operations.

jlvdb commented 2 years ago

Good points, Alex and Sam, I will check TXPipe and talk to Joe if necessary. I'll try to use as much existing infrastructure as possible unless that requires changing too much of the existing code base.