devanshkv / fetch

A set of deep learning models for FRB/RFI binary classification.
GNU General Public License v3.0
42 stars 32 forks source link

Problem with example fil files #9

Closed orajwade19 closed 4 years ago

orajwade19 commented 4 years ago

The three example fil files throw an error when used with heimdall-astro (I tried using heimdall for verification)

devanshkv commented 4 years ago

Can you please heimdall with the -G option and share the output?

orajwade19 commented 4 years ago

Here's the output:

heimdall -f 28.fil -G processing beam 1 allocating filterbank data vector for 262144 samples with size 176160768 bytes Allocating GPU... Process 0 using GPU 0 Process 0 setting CPU to spin nchans = 336 dt = 0.00126647 f0 = 1465 df = -1 Creating dedispersion plan... Initialisation complete. Using Thrust v1.9.1 Beginning data processing, requesting 262144 samples Executing pipeline on new gulp of 5120 samples... nsamp_gulp=262144 overlap=0 nsamps_read=5120 nsamps_read+overlap=5120 Cleaning 0-DM filterbank... Applying manual killmasks Bad channel count = 0 Generating DM list... dm_min = 0 dm_max = 1000 dm_tol = 1.25 dm_pulse_width = 40 nchans = 336 dt = 0.00126647 dedisp nchans = 336 dedisp dt = 0.00126647 dedisp f0 = 1465 dedisp df = -1 DM List for 0 to 1000 0 2.97719 5.95407 8.93089 11.9079 14.8853 17.8633 20.8422 23.8223 26.8037 29.7867 32.7715 35.7584 38.7476 41.7393 44.7338 47.7314 50.7321 53.7364 56.7444 59.7563 62.7725 65.7931 68.8184 71.8485 74.8839 77.9246 80.971 84.0233 87.0816 90.1463 93.2176 96.2958 99.381 102.474 105.574 108.682 111.797 114.922 118.055 121.196 124.347 127.507 130.676 133.855 137.044 140.244 143.454 146.674 149.906 153.149 156.404 159.67 162.948 166.239 169.542 172.858 176.187 179.529 182.885 186.255 189.639 193.037 196.45 199.877 203.32 206.778 210.252 213.742 217.248 220.771 224.31 227.867 231.44 235.032 238.641 242.268 245.914 249.578 253.262 256.965 260.687 264.429 268.192 271.974 275.778 279.602 283.448 287.316 291.205 295.116 299.05 303.007 306.987 310.991 315.018 319.069 323.144 327.245 331.37 335.52 339.696 343.898 348.127 352.381 356.663 360.972 365.309 369.673 374.066 378.487 382.938 387.417 391.926 396.465 401.035 405.635 410.266 414.928 419.622 424.348 429.107 433.898 438.723 443.581 448.473 453.4 458.361 463.357 468.388 473.456 478.559 483.699 488.877 494.091 499.343 504.634 509.963 515.331 520.739 526.186 531.674 537.202 542.772 548.383 554.035 559.731 565.469 571.25 577.075 582.944 588.858 594.817 600.821 606.872 612.969 619.112 625.303 631.542 637.83 644.166 650.551 656.986 663.472 670.008 676.595 683.234 689.926 696.67 703.468 710.319 717.225 724.186 731.202 738.274 745.403 752.588 759.832 767.133 774.493 781.912 789.392 796.931 804.532 812.194 819.919 827.706 835.556 843.471 851.45 859.494 867.604 875.78 884.023 892.334 900.714 909.162 917.68 926.268 934.927 943.657 952.46 961.336 970.285 979.308 988.407 997.581 1006.83 Scrunch factors: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 dm_count = 208 max delay = 1047 nsamps_computed = 4073 Allocating memory for pipeline computations... Dedispersing for DMs 0 to 1006.83... Beginning inner pipeline... dm_idx = 0 scrunch = 1 cur_nsamps = 4073 dt0 = 0.00126647 cur_dt = 0.00126647 Baselining and normalising each beam... Filtering each beam at width of 1 filter_idx=0 terminate called after throwing an instance of 'thrust::system::system_error' what(): parallel_for failed: invalid configuration argument Aborted (core dumped)

devanshkv commented 4 years ago

Try heimdall -f 28.fil -dm 10 900.

orajwade19 commented 4 years ago

Alright, that seems to be working. Could you tell me why it doesn't work with the default options? Thanks a lot.

devanshkv commented 4 years ago

I do not know for sure. I may be related to the length of the file and the dispersion delay corresponding to the maximum DM you are asking heimdall to search for.

orajwade19 commented 4 years ago

Okay, I'll explore a bit more. That said, I've checked the Heimdall output, and it is: 13.2717 2049 2.59499 9 13 48.8087 5 2048 2066
whereas the example shows: 16.8128 1602 2.02888 1 127 475.284 22 1601 1604 That's a problem, yes?

devanshkv commented 4 years ago

Yes, that's because heimdall is trying to search till the 4096 sample boxcar width, there are only ~5K samples in the file, so its only searching ~30 samples. Try heimdall -f 28.fil -G -dm 10 900 -boxcar_max 16. That should give you the FRB.

orajwade19 commented 4 years ago

Yep, that does the trick. Could you tell me HOW you determine the maximum boxcar width (and also other options for future searches)? Even a pointer to some reference/text would be much appreciated. Thanks a ton.

devanshkv commented 4 years ago

For the maximum boxcar width, say you want to search up to 50ms. This is good enough number given most FRBs have been < 35ms. The boxcar width would be 50ms/sampling_interval. Heimdall only searches in powers of two, so you can round it up to the next power of two.

orajwade19 commented 4 years ago

Thanks a lot for the quick reply. Should I close the issue, or keep it open for others to refer to? Also, if you could, are there any resources for learning a bit about FRB search techniques (at least sufficient enough to properly use Heimdall ) ? Again, appreciate the quick response.

devanshkv commented 4 years ago

Happy to help.

There is some summary in Scott's talk and on Shami's page. You can also take a look at the recent paper by the AstroAccelerate folks.