Noise simulations generated from TOD sims

[msyriac]

In the recent effort to inform LAT scan strategies through TOD simulations, @zatkins2 @AdriJD and others have produced models for generating map-based simulations. These are much more realistic and should replace the simplistic model. It would be good to build an interface to these models, if this has not already been done.

cc: @keskitalo @smsimon

[zatkins2]

Thanks for bringing this up, Mat. We should try to have one solution that abstracts away as much of the awareness of the products themselves (i.e. works seamlessly for both the TOD simulations and actual data once available). I think there are several layers to this issue and I can update my knowledge of their status. Some questions and

• [ ] to-dos

are sprinkled throughout. Please add further comments!

---

Low Level

• TOD simulations and data (a.k.a. model inputs) are served by soapack (is that still the plan for the data?)
  • Adri has started a soapack branch, so_scan_s0002 that supports the latest TOD simulations.
• [ ] If soapack is still the way forward, we should make sure all filenames etc. conform to your data policy and provide the relevant SO DataModels in the main branch.

Intermediate Level

• The code (mnms) for generating models from low level data only relies on the above functionality.
• mnms has a ready-made interface in going from models to simulations.
  • The interface in supplying an on-disk vs. on-the-fly simulation is identical (users only notice a speed difference :p)
• mnms manages its own data products internally via an entry in a user's soapack.yml config.
  • This includes both models and simulations.
  • Models for the latest scan strategies already exist (@AdriJD would know where?).
• [ ] What is workflow for generating models in the future (new TOD sims, as data is released)? Answering this is necessary before thinking about whether any additional interface (i.e., beyond that in mnms already) might be needed.

High Level

• [ ] Ideally, any simpler noise models implemented already in this repo should be moved into mnms (i.e., implement a concrete mnms.noise_models.NoiseModel class)
• [ ] I don't know what the current plan is for user interface with all SO sims, but for supplying the noise part of that, likely only a thin wrapper around mnms.noise_models.NoiseModel.get_sim is needed.
• [ ] Hosting mnms in simonsobs would be a good start heh (I can do this soon-ish). I would like to keep it as a separate repo akin to pysm, and let this repo be mainly for interface to specific products.

[msyriac]

Very good idea to abstract together current sims and future data, strongly in agreement. Some thoughts:

• I wish we had a better alternative to soapack, but until then, let's assume it is the way forward.
• Shall we start with whatever filename convention @keskitalo and others have used for the scan sims? I can work the future proposal/policy around that.
• In the future, we want each map release to be "automatically" accompanied by a map-based sim model. A utopian scenario would be that map-maker scripts have a plug-in that fires up a job to create the mnms noise models once the maps, ivars, etc. are saved. (Probably realistically after source-subtracted maps are available). In the meantime, imagine that this firing up is done by-hand through a script submission that points to the necessary inputs. (What is the current workflow on ACT?)
• Moving simpler noise models to mnms is good, but can be low priority: they have not been used much and likely won't be needed heavily in the future.

[zonca]

@zatkins2 where can I find instructions to load the latest tiled noise simulations for SO?

[zatkins2]

@msyriac sounds good on soapack and filenames. I guess I meant filenames for the future data as well! The workflow for ACT has been: see if any changes are needed in soapack to accommodate new map products + update soapack, see if any other metadata should be updated (masks, source catalogs, etc), run models (one script) and evaluate them manually (another script, basically look at maps and PS ratios). And agreed on the simpler models. So yes, I think the best path is hosting mnms on simonsobs, wrapping its interface in some more unified SO mbs interface, and making sure soapack can load TOD sims and SO maps.

@zonca Adri ran these for the SO TOD sims, info on location is here https://simonsobs.slack.com/archives/CDEFD39GT/p1649969971165499. There are also "wavelet" and "directional wavelet" models, but I'm not sure where they are (@AdriJD can confirm, probably same location!). The interface for loading them is identical other than changing the subclass of your mnms.noise_models.NoiseModel instance.

[AdriJD]

@zonca The link that Zach posted would tell you where the tiled noise models are stored. The sims that are drawn from these models are not stored in a shared space I believe. Frank generated them from the models and stored them here: nersc:/global/homes/j/jia_qu/maps/so_scan; not sure if they're still there, NERSC seems to be down right now.

For the directional wavelet models we have the opposite situation, the sims are stored here: nersc:/global/homes/a/adriaand/scratch/so/20220630_scan_s0002_fdw but the models are not on NERSC. I can transfer them if needed. Loading these sims from within mnms should be done using:

from soapack import interfaces
from mnms import noise_models as nm

data_model = interfaces.SOScanS0002()

model = nm.FDWNoiseModel(qid, data_model=data_model)

sim_idx = 10 # For 11th sim.
split_idx = 0 # For split 1 of 2.
alm = model.get_sim(split_idx, sim_idx, check_on_disk=True, alm=True, write=False)

Note you have to be on the so_scan_s0002 branch of soapack + use the same .soapack.yml file as specified in the instructions linked by Zach.

Sorry about the messy state of things. What's your short term goal with the sims?

[msyriac]

The short term goal here is to provide a set of noise simulations of SO to the AWGs that they can use to test pipelines. We can have a dozen or so saved on disk and accessible through whatever interface @zonca has already made available. And there should be some on-the-fly generation interface too that is compatible with @zonca's previous work. This is one of the situations where I would advocate patching together something quick rather than making it too "future-proof".

[zonca]

I have never written any interface to load maps already available on disk. What I have in mapsims is just for creating noise realizations on the fly, it starts from a configuration file which is used to configure a MapSim object, which is then executed to get the maps, see the interface in this notebook:

https://gist.github.com/zonca/5d920fe8269831e0d1aecd88ec7d9d50

which is part of this release of map-based noise simulations (just from a simple spectrum):

https://github.com/simonsobs/map_based_simulations/blob/master/202006_noise/README.md#generate-other-realizations-on-the-fly

How computationally intensive is the generation on-the-fly? if it is not too heavy, we could just provide that instead of access to the maps.

I would like to take a peek at the maps, however I get permission denied at /global/homes/j/jia_qu/maps/so_scan

[zonca]

@AdriJD any update on this topic?

[zatkins2]

I think my initial suggestion is still the best path forward: a simple wrapper around the mnms.noise_models.NoiseModel.get_sim method. The configuration for mnms and the code itself does all the work of loading a sim from disk, or generating one on the fly instead, for free. You select which you want to do just with arguments to get_sim (on the fly is slow enough that you want to be able to load from disk if possible). I think then all that is needed for @msyriac 's short term goal is just moving the scan strategy model and sim files to a public place (this would be for @AdriJD and @qujia7 to look at when they are free), and writing this minimal wrapper (which I would do). When is this needed by?

[AdriJD]

Hi @zonca, sorry about the delay, I am back to work now. I have asked Frank to change the permissions for the folder.

If we focus on the scan strategy noise sims for now (which are based on the only SO TOD sim we have at the moment), I think that Zach's plan is indeed the way to go. We don't want to reinvent the wheel for the I/O routines needed to load up the existing models/maps. Another advantage of Zach's proposal is that the wrapper around the mnms code would be the same regardless of whether we use the existing maps or go for on-the-fly generation. I expect most work will go into creating an easy way for people to set up soapack, but as long as we restrict ourselves to NERSC for now that should be fine.

I will transfer the model to NERSC and then perhaps Zach and I figure out a soapack config that works (we should also still merge that SO scan branch in soapack I think). If we then present a working mnms interface to the models/maps we should start on the wrapper.

Let's perhaps also discuss this briefly at the PWG call on Thursday.

[AdriJD]

@zonca, Frank's files should be visible now.

[zonca]

I'm trying to load the maps with soapack but I get an error: https://gist.github.com/9ae91c3929cda926110958a395b98a18 Anyway, I think it is not very useful to debug this, I'll try again once you release the new set of maps.