WCS sampling & FITS to TOAST FITS

[imbasimba]

Mostly addresses https://github.com/WorldWideTelescope/toasty/issues/81, but does nothing about masked pixels and does not yet support a collection of FITS input.

The main feature is based around the WCS sampler & filter, which allows us to create a TOAST imageset from any array/wcs input. The tile_fits method now has the ability to create TOAST imagesets, which also has replaced HiPS as the default tiling method for images covering a large area of the sky.

Upgrades to the general process:

1. toast_base does now accept tile filters to allow processing of a specific area, and skipping every tile outside input bounds
2. Same for the cascade process
3. Replaced the parameters force_tan etc. with a TilingMethod enum

Requires https://github.com/WorldWideTelescope/wwt_data_formats/pull/50 to work properly.

[codecov[bot]]

Codecov Report

Merging #86 (e9a8774) into master (80a3976) will increase coverage by 1.91%. The diff coverage is 84.77%.

@@            Coverage Diff             @@
##           master      #86      +/-   ##
==========================================
+ Coverage   75.42%   77.34%   +1.91%     
==========================================
  Files          23       23              
  Lines        3520     3703     +183     
==========================================
+ Hits         2655     2864     +209     
+ Misses        865      839      -26     

Impacted Files	Coverage Δ
toasty/cli.py	80.42% <0.00%> (ø)
toasty/collection.py	56.66% <0.00%> (-0.64%)	:arrow_down:
toasty/fits_tiler.py	46.93% <70.00%> (-0.40%)	:arrow_down:
toasty/merge.py	92.81% <88.40%> (-2.59%)	:arrow_down:
toasty/samplers.py	87.57% <88.43%> (+13.62%)	:arrow_up:
toasty/pyramid.py	94.47% <93.33%> (-0.38%)	:arrow_down:
toasty/__init__.py	100.00% <100.00%> (ø)
toasty/builder.py	96.73% <100.00%> (+0.04%)	:arrow_up:
toasty/image.py	86.07% <0.00%> (+0.20%)	:arrow_up:
... and 2 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[imbasimba]

Now that the tests are properly creating a TOAST from FITS, they are failing because they are not yet using the latest wwt_data_formats https://github.com/WorldWideTelescope/wwt_data_formats/pull/50

[pkgw]

wwt_data_formats 0.15.1 is now out with the new code. Closing and opening to re-trigger the CI.

[pkgw]

OK, I have pushed a couple of commits that I think fix the computation of image bounds, and it turns out that the lat/lon sampler also needed fixing to deal with images spanning very large amounts of latitude. I now can fully TOAST the DASCH test image (which I'm doing after reprojecting it to a straight TAN projection to avoid the Astropy WCS bug). They're not fully tested, but I wanted to get them uploaded before I go away for my travel.

Unfortunately the positioning of the image appears to be off after these fixes. I need to look into why that's happening.

[pkgw]

Good catch on the min/max mistake, thanks!

[pkgw]

OK, I've pushed one more commit that avoids a crash in hipsgen with my DASCH test file.

The "good" news is that my test image is landing in the wrong place because the reproject package is doing something very wrong when converting it from TPV to TAN projection, as far as I can tell. So that's not Toasty's fault.

edit: the reproject problem seems to be a consequence of https://github.com/astropy/astropy/issues/13509 . The image shape looks reasonable, but the image is half-empty.

However, the less-good news is that if I compare the TOAST and hipsgen output, there's still a disagreement. It looks like there's a negated rotation term somewhere. But I wouldn't worry about this too much since the image coordinate system may be generally messed up.

[pkgw]

For posterity, here's what I'm seeing. The tnx file is TPV, the tanproj file is reprojected by reproject. The colorscales labeled on the left indicate which image is landing where. The view is pretty much centered on the North Pole.

[pkgw]

Here's hipsgen vs. TOAST of the TPV file — looks like the AstroPy bug didn't arise here. It's close, but rotated 180°, or something similar (note the area with writing on it, in the two images). The hipsgen version is correct.

[imbasimba]

I had planned to add handling of FITS collections to a new PR, but it turned out this functionality could be added with minimal changes (thanks past toasty developers!).

This may not be the most optimal way of toasting multiple FITS files, but it seems to work quite well. Let's discuss here or on Monday @pkgw

[pkgw]

I figured out the problem with the DASCH test image, and it appears to be something outside of all of the TOAST infrastructure: the file should have a LONPOLE header. Based on my reading of Calabretta & Greisen 2002, the headers in the file are not correct. Forcing LONPOLE to 180 fixes the celestial alignment for this test image.

[pkgw]

OK, there is still some more work to do here — in the DECaPS2 test processing I am running into some scalability issues since there are about 28 million tiles to process. But the code here seems to basically be working reliably, and I think it makes the most sense to close out this PR and continue evolving in separate ones.

Docs build is failing here due to an expired cert for dotnetfoundation.org. Merging anyway.