search

ovrocaltech / ovro-alert

Code and services for sending, receiving, and using astronomical alerts at OVRO
BSD 3-Clause "New" or "Revised" License
1 stars 0 forks source link

added lwa buffer triggering #11

Closed nkosogor closed 1 year ago

nkosogor commented 1 year ago

@jack-h can you take a look at the code with buffer triggering in buffer_trigger.py, which I wrote based on one of our zooms?

nkosogor commented 1 year ago

@jack-h, can you please look at this part below? Is it right that if I want to use the highest frequencies I should trigger the pipelines with IDs 2 and 3? https://github.com/ovrocaltech/ovro-alert/blob/25ec16f52a7a2eaf9048d37d03850e71403bb03e/ovro_alert/lwa_alert_client.py#L9 And also I'm sending the data to /data0 and /data1 with corresponding IDs. Is it correct? https://github.com/ovrocaltech/ovro-alert/blob/25ec16f52a7a2eaf9048d37d03850e71403bb03e/ovro_alert/lwa_alert_client.py#L28-L35

jack-h commented 1 year ago

TL;DR -- yes, I think you are correct on both counts

For my info more than yours, probably...

Mapping from the interleaved config file:

# lwa-gpu01
    10.41.0.65-10000: [ 560,  656]
    10.41.0.18-10000: [ 656,  752]
    10.41.0.65-20000: [2096, 2192]
    10.41.0.18-20000: [2192, 2288]
    # lwa-gpu02
    10.41.0.66-10000: [ 752, 848]
    10.41.0.17-10000: [ 848, 944]
    10.41.0.66-20000: [2288, 2384]
    10.41.0.17-20000: [2384, 2480]
...

Higher freq channels go to the X-engine NICs on port 20000.

From https://github.com/realtimeradio/caltech-bifrost-dsp/blob/a9cda5bcca08d2e41d54efa931e078a20134c65e/pipeline-control/lwa352_pipeline_control/lwa352_xeng_etcd_client.py#L18 these are indeed the last two pipelines. (IDs 2, 3 of 0,1,2,3).

Assuming (as is the intended setup) that /data0 is associated with the NUMA node including CPU0, and /data1 with the NUMA node including CPU 16, then pipeline 2 should write to data0 and pipeline 3 to data1. (Based on https://github.com/realtimeradio/caltech-bifrost-dsp/blob/a9cda5bcca08d2e41d54efa931e078a20134c65e/pipeline-control/lwa352_pipeline_control/lwa352_xeng_etcd_client.py#L22)

jack-h commented 1 year ago

PS -- RAM buffer size is currently only set to 16GB (https://github.com/realtimeradio/caltech-bifrost-dsp/blob/a9cda5bcca08d2e41d54efa931e078a20134c65e/pipeline-control/lwa352_pipeline_control/lwa352_xeng_etcd_client.py#L20) but can be updated whenever, or made dynamic, if desired. It's currently relatively small because otherwise you have to wait a while for all the RAM to be allocated before the pipeline can start