Distance marg

[dwysocki]

This contains a number of fixes and optimizations to the fast distance marginalization code. We've been able to run the code with the following command in the ILE-GPU-Paper/demos/test_workflow_batch_gpu_lowlatency/ test case that's generated by Travis-CI. However, the results seem off, so there may be a remaining bug or we're calling the code incorrectly. Note that it only runs if --gpu and --force-xpy are provided, though it will still default to numpy if cupy is not installed.

/home/dwysocki/.virtualenvs/RIFT-dist-marg/bin/integrate_likelihood_extrinsic --n-chunk 8000 --time-marginalization --sim-xml overlap-grid.xml.gz --reference-freq 100.0 --adapt-weight-exponent 0.1 --event-time 1000000014.236547946 --save-P 0.1 --cache-file /home/dwysocki/research/research-projects-RIT/ILE-GPU-Paper/demos/zero_noise.cache --fmin-template 10 --n-max 20000000 --fmax 1700.0 --save-deltalnL inf --l-max 2 --n-eff 200 --approximant SEOBNRv4 --adapt-floor-level 0.1 --d-max 1000 --psd-file H1=/home/dwysocki/research/research-projects-RIT/ILE-GPU-Paper/demos/HLV-ILIGO_PSD.xml.gz --psd-file L1=/home/dwysocki/research/research-projects-RIT/ILE-GPU-Paper/demos/HLV-ILIGO_PSD.xml.gz --channel-name H1=FAKE-STRAIN --channel-name L1=FAKE-STRAIN --inclination-cosine-sampler --declination-cosine-sampler --data-start-time 1000000008 --data-end-time 1000000016 --inv-spec-trunc-time 0 --no-adapt-distance --vectorized --gpu --force-xpy  --sim-xml overlap-grid-0.xml.gz --save-samples --output-file samples.xml -d -l distance_marginalization_lookup.npz

A brief summary of changes:

• full cupy support
• fixes to lookup table accuracy at high SNR using alternative form of integral devised by @SMorisaki [1]
• improved interpolation coordinate system devised by @SMorisaki [1]
• better use of vectorization
• general fixes needed to actually run the code
  • this includes adding dummy distance samples (all taking the reference value) to the sampler object after it's done
  • this was done to avoid crashing
  • if there is another workaround that doesn't involve adding dummy samples we should switch to that -- this will be a problem for other extrinsic marginalization improvements as well

[1] An implementation of distance marginalization LIGO-T2100485