Change binning for Fourier space measurements

[jpratmarti]

I think it would be nice to be able to change the ell binning (range and number of bins) from the config file, but I'm not sure how to do it since at the moment they are constructed like:

ell_bins = MyNmtBin(int(pixel_scheme.nside), nlb=nlb)

[joezuntz]

MyNmtBin is a subclass of NmtBin, which has a method from_edges(ell_ini, ell_end) where those two are arrays of the start and end points of each bin.

[damonge]

There are a bunch of new constructors in namaster (and happy to implement more): https://namaster.readthedocs.io/en/latest/pymaster.html#pymaster.bins.NmtBin.from_edges

(sorry I only saw this now)

[jpratmarti]

Thank you @damonge. Getting back to this now, today in the hack session we discussed adding these options in the config file: "ell_min": 10, "ell_max": 1000, "n_ell": 20, "ell_spacing": 'log' Does this make sense to you? Also, then using these options as a starting point, do you know how we could use the function you're point me to? Inside the choose_ell_bins function, we could replace the line: #ell_bins = MyNmtBin(int(pixel_scheme.nside), nlb=nlb) with the line: ell_bins = MyNmtBin.from_edges(self.config['ell_min'], self.config['ell_max']) but I'm not sure if this is what the from_edges functions is asking for exactly, since it's supposed to take an array with the lower edges of each bandpower. How would "ell_min" relate to the bandpowers?

[joezuntz]

It looks like from_edges wants two arrays, one of ell_min and one of ell_max, each per bin, so I think you'd need to generate all the edges using edges = np.geomspace(ell_min, ell_max, n_ell) and then take edges[:-1] as the mins and edges[1:] as the maxs. You might need to convert to integers also.

(np.geomspace is like np.logspace but you don't have do the log of the first two args)

[jpratmarti]

Thanks a lot @joezuntz for the detailed answer, this is very useful. I was mostly confused about whether bandpowers were directly related to the ells, but if they are the same actually that will be easy. Thanks again.

[jpratmarti]

When running the code I"m getting this error below. It seems like the version installed (in ceci?) is not reconizing the 'is_Dell' option. I tried to install pymaster separately with pip but I get an error on the installation. I'm not sure if this is a problem of my installation or related to ceci.

/global/cscratch1/sd/jprat/TXPipe/txpipe/twopoint_fourier.py:628: UserWarning: Using unweighted lens samples here
  warnings.warn("Using unweighted lens samples here")
Exception ignored in: <bound method NmtBin.__del__ of <txpipe.utils.nmt_utils.MyNmtBin object at 0x2aaab16bb588>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/pymaster-1.0.2-py3.6-linux-x86_64.egg/pymaster/bins.py", line 158, in __del__
    if self.bin is not None:
AttributeError: 'MyNmtBin' object has no attribute 'bin'
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/global/cscratch1/sd/jprat/TXPipe/txpipe/__main__.py", line 5, in <module>
    PipelineStage.main()
  File "/usr/local/lib/python3.6/dist-packages/ceci/stage.py", line 269, in main
    stage.execute(args)
  File "/usr/local/lib/python3.6/dist-packages/ceci/stage.py", line 351, in execute
    stage.run()
  File "/global/cscratch1/sd/jprat/TXPipe/txpipe/twopoint_fourier.py", line 112, in run
    ell_bins = self.choose_ell_bins(pixel_scheme, f_sky)
  File "/global/cscratch1/sd/jprat/TXPipe/txpipe/twopoint_fourier.py", line 458, in choose_ell_bins
    ell_bins = MyNmtBin.from_edges(edges_int[:-1], edges_int[1:], is_Dell=False)
  File "/usr/local/lib/python3.6/dist-packages/pymaster-1.0.2-py3.6-linux-x86_64.egg/pymaster/bins.py", line 155, in from_edges
    is_Dell=is_Dell)
TypeError: __init__() got an unexpected keyword argument 'is_Dell'

[joezuntz]

Might be to do with my subclassing NmtBin.

[jpratmarti]

ok @joezuntz, thanks. Do you have any suggestions for fixing this?

[joezuntz]

You can make this change in nmt_utils.py

-    def __init__(self, nside, bpws=None, ells=None, weights=None, nlb=None, lmax=None):
-        super().__init__(nside, bpws=bpws, ells=ells, weights=weights, nlb=nlb, lmax=lmax)
+    def __init__(self, nside, bpws=None, ells=None, weights=None, nlb=None, lmax=None, is_Dell=False, f_ell=None):
+        super().__init__(nside, bpws=bpws, ells=ells, weights=weights, nlb=nlb, lmax=lmax, is_Dell=False, f_ell=None)

I'll let you do it in your branch to avoid clashes.

[jpratmarti]

Ok thanks Joe. I tried it and it fixes something but doesn't quite work. There is this error:

/global/cscratch1/sd/jprat/TXPipe/txpipe/twopoint_fourier.py:628: UserWarning: Using unweighted lens samples here
  warnings.warn("Using unweighted lens samples here")
Exception ignored in: <bound method NmtBin.__del__ of <txpipe.utils.nmt_utils.MyNmtBin object at 0x2aaab36e4630>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/pymaster-1.0.2-py3.6-linux-x86_64.egg/pymaster/bins.py", line 158, in __del__
    if self.bin is not None:
AttributeError: 'MyNmtBin' object has no attribute 'bin'
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/global/cscratch1/sd/jprat/TXPipe/txpipe/__main__.py", line 5, in <module>
    PipelineStage.main()
  File "/usr/local/lib/python3.6/dist-packages/ceci/stage.py", line 269, in main
    stage.execute(args)
  File "/usr/local/lib/python3.6/dist-packages/ceci/stage.py", line 351, in execute
    stage.run()
  File "/global/cscratch1/sd/jprat/TXPipe/txpipe/twopoint_fourier.py", line 112, in run
    ell_bins = self.choose_ell_bins(pixel_scheme, f_sky)
  File "/global/cscratch1/sd/jprat/TXPipe/txpipe/twopoint_fourier.py", line 458, in choose_ell_bins
    ell_bins = MyNmtBin.from_edges(edges_int[:-1], edges_int[1:], is_Dell=False)
  File "/usr/local/lib/python3.6/dist-packages/pymaster-1.0.2-py3.6-linux-x86_64.egg/pymaster/bins.py", line 155, in from_edges
    is_Dell=is_Dell)
TypeError: __init__() missing 1 required positional argument: 'nside'

Maybe I need to inicialize this class first? pymaster.bins.NmtBin

[joezuntz]

Think you need to change the two nsides to nside=None and nside=nside in the MyNmtBin init method (in both places).

[jpratmarti]

Great, this worked thanks so much!

[jpratmarti]

Ok, I managed to get the twopoint measurements, but if I try to set ell_max >1500 it crashes because it says the maps have not enough resolution. Are we expecting to increase the resolution at some point or are we fine with this ell_max? Just for completeness and future reference I will post the error here:

/global/cscratch1/sd/jprat/TXPipe/txpipe/twopoint_fourier.py:628: UserWarning: Using unweighted lens samples here
  warnings.warn("Using unweighted lens samples here")
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/global/cscratch1/sd/jprat/TXPipe/txpipe/__main__.py", line 5, in <module>
    PipelineStage.main()
  File "/usr/local/lib/python3.6/dist-packages/ceci/stage.py", line 269, in main
    stage.execute(args)
  File "/usr/local/lib/python3.6/dist-packages/ceci/stage.py", line 351, in execute
    stage.run()
  File "/global/cscratch1/sd/jprat/TXPipe/txpipe/twopoint_fourier.py", line 114, in run
    workspaces = self.make_workspaces(maps, calcs, ell_bins)
  File "/global/cscratch1/sd/jprat/TXPipe/txpipe/twopoint_fourier.py", line 416, in make_workspaces
    spaces[(i,j,k)] = get_workspace(f1, f2)
  File "/global/cscratch1/sd/jprat/TXPipe/txpipe/twopoint_fourier.py", line 396, in get_workspace
    space.compute_coupling_matrix(f1, f2, ell_bins,is_teb=False, n_iter=1)
  File "/usr/local/lib/python3.6/dist-packages/pymaster-1.0.2-py3.6-linux-x86_64.egg/pymaster/workspaces.py", line 74, in compute_coupling_matrix
    lmax_mask)
RuntimeError: Requesting bandpowers for too high a multipole given map resolution