Index built from UCAC4?

[dstndstn]

From Doug Welch:

Have you or anyone you know prepared UCAC4-based indices for astrometry.net? If so, would it be possible to get a link?

[dstndstn]

Nope, not that I know of. I experimented with UCAC3 (as evidenced by blind/ucac3tofits.c), so it probably wouldn't take much to adapt to UCAC4. Index-building is pretty easy these days.

[DougWelch]

Thanks for the quick reply.

I did see that it is possible to produce one's own indices but I have not yet done so.

Are the UCAC3 indices accessible somewhere?

[dstndstn]

Thanks for carrying on this conversation here. [Edited in response to some errors Doug found]

I don't think I ever created indices from UCAC3 -- I think I was just parsing it when I was looking for offsets between different astrometric catalogs.

It looks like ucac3tofits wants to split the catalog into healpix tiles, so you'll probably need something like this:

ucac3tofits -N 1 /path/to/ucac3/z???.bz2

which should produce 12 files named like ucac3_000.fits up to 011.fits

Then split them into (slightly overlapping) healpix tiles like this:

hpsplit -o split-%02i.fits -n 1 -m 1 ucac3_???.fits

for a 1-degree overlap margin (-m 1), and splitting the sky into 12 pieces (level-1 healpix grid, -n 1)

That will produce a bunch of split-00.fits files.

At this point you might want to trim out any FITS table columns you don't want, because they will get copied into the index files and make the files huge; something like,

fitscopy split-00.fits"[col RA;DEC;magm]" cut-00.fits

Then run build-index on each of those "cut-??.fits" files, for each desired scale. Choose an index ID (I often encode the date and scale, eg 13010202 for 2013-01-02, scale 02) for each.

build-index -i cut-00.fits -o index-ucac-02-00.fits -P 2 -S magm -H 0 -s 1 -I 13010202

-P is the "preset" scale -- scale 2 is features 4 to 5.6 arcmin in diameter (appropriate for solving SDSS fields); UCAC4 has only ~100 million stars so that scale might be too small (ie, require more stars than are available). You'll see that it's running out of stars if the number of stars found per sweep drops a lot.

-S tells it which column to sort on -- ie, use the "magm" -- UCAC magnitude -- column to prefer bright stars.

-H 0 tells it to mark this index as being healpix tile 0,

and -s 1 tells it to mark that as healpix scale Nside=1.

-I is the index id.

To be clear, if the healpix tile is HH and the scale is SS, then the command should be,

build-index -i cut-${HH}.fits -o index-ucac-${SS}-${HH}.fits -P ${SS} -S magm -H ${HH} -s 1 -I 130102${SS}

Then to use the files, you'll need to create a "backend config" file to tell the solver where to find them;

solve-field --backend-config /path/to/ucac4.cfg

Where that ucac4.cfg file might contain something like,

add_path /path/to/ucac4/index/files autoindex inparallel

Hope that helps, --dstn

[DougWelch]

Hi Dustin,

The hpsplit segfaulted, apparently due to the size of ucac3_007.fits which was greater than 2GB. As per your suggestion, I did the fitscopy first to reduce the size of the created files like so:

fitscopy ucac3_000.fits"[col RA ; DEC ; MAG]" ucac3less-000.fits ... fitscopy ucac3_011.fits"[col RA ; DEC ; MAG]" ucac3less-011.fits

Then, I did the hpsplit like so: hpsplit -o split-%03i.fits -n 1 -m 1 ucac3less-???.fits which appeared to succeed, as did: build-index -i ucac3less-000.fits -o index-ucac-02-000.fits -P 2 -S MAG -H 0 -s 1 -I 13010202

But the next attempt did not:

build-index -i ucac3less-002.fits -o index-ucac-02-002.fits -P 2 -S MAG -H 0 -s 1 -I 13010202 Reading ucac3less-002.fits... Got 2025027 stars Sweep 1: 0 stars Sweep 2: 0 stars Sweep 3: 0 stars Sweep 4: 0 stars Sweep 5: 0 stars Sweep 6: 0 stars Sweep 7: 0 stars Sweep 8: 0 stars Sweep 9: 0 stars Sweep 10: 0 stars Total: 0 stars Writing output... fitstable.c:913:read_array_into: Failed to read column from FITS file startree2.c:122:startree_build: Failed to read RA from column RA build-index.c:530:build_index: Failed to create star kdtree

Am I correct in understanding that I should be going through all 12 of the ucac3less FITS files in this way?

One other issue that is worth mentioning is that the images I will be using these indices on are 2.9x2.9 degrees and the stars used to determine the WCS will likely be mag 15 or brighter.

Thanks again for your help and guidance! Doug

[dstndstn]

Ok, great, that makes sense: the "-H" arg must match the healpix number; you want to tell build-index that it's supposed to be indexing the part of the sky that your catalog actually covers :)

build-index -i ucac3less-002.fits -o index-ucac-02-002.fits -P 2 -S MAG -H 0 -s 1 -I 13010202

should be ("-H 2" not "-H 0")

build-index -i ucac3less-002.fits -o index-ucac-02-002.fits -P 2 -S MAG -H 2 -s 1 -I 13010202

And, for 3x3-degree fields, you'll probably want scales more like 8-10.

Yes, you should loop over the scales and healpixes, building one index for each.

cheers, --dstn

[DougWelch]

Hi Dustin,

I appear to have successfully produced a set of indices with the following csh script:

!/bin/tcsh

foreach HH (00 01 02 03 04 05 06 07 08 09 10 11) foreach SS (08 09 10 11 12 13) build-index -i ucac3less-0${HH}.fits -o index-ucac3-${SS}-${HH}.fits -P ${SS} -S MAG -H ${HH} -s 1 -I 130103${SS} end end

I will test these out and, if they are working as expected, I will make them available.

Thanks for your help!

Cheers, Doug