Closed ljgray closed 2 years ago
Although the products aren't quite right yet, this reduces the RFIFilter
step down to ~15 seconds in a daily pipeline run
Depends on radiocosmology/caput#211
Interestingly, the mask being produced is different from that produced by the previous method.
@jrs65 this produces the same mask now as previous revisions, so it should be good to go unless there are additional changes.
In its current implementation, the RFI Filter step of the daily pipeline takes ~25 minutes. Most of this time is spent taking a rolling median, and it is done in a single process. This PR should address the issue by pre-calculating a median across frequencies to flatten the auto vis. data, then distributing across the frequency axis to distribute the computation across processes.