Speed up Issue with HAL

[omodei]

Feature Request

During the HAWC collaboration meeting in Puerto Vallarta, it was noted that 3ML analysis was much slower with respect gammapy analysis. Quentin looked into possible bottleneck of the HAL code. Here is a summary (verbatim from him):

• The PSF for extended sources is taken at the center of the ROI and not at the position of each source, could be a problem for large ROIs: https://github.com/search?q=repo%3AthreeML/hawc_hal%20point_source_image&type=code https://github.com/threeML/hawc_hal/blob/ce740382a5c6cb680f8a05ac9ac73690633da172/hawc_hal/convolved_source/convolved_point_source.py#L59
• The PSF convolutor for extended source uses the default psf_integration_method=‘exact’ while for point source it is set to fast https://github.com/threeML/hawc_hal/blob/ce740382a5c6cb680f8a05ac9ac73690633da172/hawc_hal/HAL.py#L209 https://github.com/threeML/hawc_hal/blob/ce740382a5c6cb680f8a05ac9ac73690633da172/hawc_hal/HAL.py#L215-L217 https://github.com/threeML/hawc_hal/blob/ce740382a5c6cb680f8a05ac9ac73690633da172/hawc_hal/psf_fast/psf_convolutor.py#L24-L25
• For extended sources the psf convolution is not cached if the source parameters do not change https://github.com/threeML/hawc_hal/blob/ce740382a5c6cb680f8a05ac9ac73690633da172/hawc_hal/HAL.py#L999-L1030
• The following should be moved to a function that return npred for each source so it can be cached if its parameters do not change, it seems that only the psf interpolation is cached https://github.com/threeML/hawc_hal/blob/ce740382a5c6cb680f8a05ac9ac73690633da172/hawc_hal/HAL.py#L974-L992
• many functions use this to compute npred but there is no caching https://github.com/threeML/hawc_hal/blob/ce740382a5c6cb680f8a05ac9ac73690633da172/hawc_hal/HAL.py#L966
• the likelihood could be evaluated on the flat geometry without need to reproject to healpix, could help if reprojection is slow https://github.com/threeML/hawc_hal/blob/ce740382a5c6cb680f8a05ac9ac73690633da172/hawc_hal/HAL.py#L1032-L1045
• this loop could be parallelized https://github.com/threeML/hawc_hal/blob/ce740382a5c6cb680f8a05ac9ac73690633da172/hawc_hal/HAL.py#L844
• this loop could be parallelized https://github.com/threeML/hawc_hal/blob/ce740382a5c6cb680f8a05ac9ac73690633da172/hawc_hal/HAL.py#L972
  • not sure to understand how the parameter change property is used but it could differentiate spatial and spectral parameters because if only spectral parameters change it is not necessary to repeat the PSF convolution, one can rescale the cached value with a different weight in each energy bin. (edited) https://github.com/threeML/hawc_hal/blob/ce740382a5c6cb680f8a05ac9ac73690633da172/hawc_hal/convolved_source/convolved_extended_source.py#L133

[omodei]

Before starting changing the code, we should keep track of our progresses. I suggest to set up a "catalog-style" example of one ROI where the convergence/speed is problematic, and work our way from there.