Replicating astrometry.net `solve-field` solving approach

[prajwel]

Hi,

I tried solving the same coordinate list (containing 260 sources) using my local astrometry.net installation (version 0.78) and pip-installed astrometry package.

Using the solve-field of astrometry.net, I get the solution within a relatively very short time:

Only searching for solutions within 0.33 degrees of RA,Dec (53.0523,-27.8858)
Field 1 did not solve (index index-5007-08.fits, field objects 1-50).
Field 1 did not solve (index index-5006-08.fits, field objects 1-50).
Field 1 did not solve (index index-5005-08.fits, field objects 1-50).
Field 1 did not solve (index index-5004-35.fits, field objects 1-50).
Field 1 did not solve (index index-5003-35.fits, field objects 1-50).
Field 1 did not solve (index index-5002-35.fits, field objects 1-50).
Field 1 did not solve (index index-5001-35.fits, field objects 1-50).
Field 1 did not solve (index index-5007-08.fits, field objects 51-100).
Field 1 did not solve (index index-5006-08.fits, field objects 51-100).
Field 1 did not solve (index index-5005-08.fits, field objects 51-100).
Field 1 did not solve (index index-5004-35.fits, field objects 51-100).
Field 1 did not solve (index index-5003-35.fits, field objects 51-100).
Field 1 did not solve (index index-5002-35.fits, field objects 51-100).
Field 1 did not solve (index index-5001-35.fits, field objects 51-100).
Field 1 did not solve (index index-5007-08.fits, field objects 101-150).
Field 1 did not solve (index index-5006-08.fits, field objects 101-150).
Field 1 did not solve (index index-5005-08.fits, field objects 101-150).
Field 1 did not solve (index index-5004-35.fits, field objects 101-150).
Field 1 did not solve (index index-5003-35.fits, field objects 101-150).
Field 1 did not solve (index index-5002-35.fits, field objects 101-150).
Field 1 did not solve (index index-5001-35.fits, field objects 101-150).
Field 1 did not solve (index index-5007-08.fits, field objects 151-200).
Field 1 did not solve (index index-5006-08.fits, field objects 151-200).
Field 1 did not solve (index index-5005-08.fits, field objects 151-200).
Field 1 did not solve (index index-5004-35.fits, field objects 151-200).
Field 1 did not solve (index index-5003-35.fits, field objects 151-200).
Field 1 did not solve (index index-5002-35.fits, field objects 151-200).
Field 1 did not solve (index index-5001-35.fits, field objects 151-200).
Field 1 did not solve (index index-5007-08.fits, field objects 201-250).
Field 1 did not solve (index index-5006-08.fits, field objects 201-250).
mo field stars:
  star 230; field_xy 1107.2,2182.7, field_orig 1107.2,2182.7
  star 9; field_xy 2417.2,4033.9, field_orig 2417.2,4033.9
  star 208; field_xy 2226.1,2369.3, field_orig 2226.1,2369.3
  star 4; field_xy 2612.5,2817.5, field_orig 2612.5,2817.5
  log-odds ratio 22.3587 (5.13144e+09), 4 match, 0 conflict, 5 distractors, 83 index.
  RA,Dec = (53.0863,-27.9144), pixel scale 0.416898 arcsec/pix.
  Hit/miss:   Hit/miss: --+---+++(best)----------+---------------------------+----------------------------------------------------
Field 1: solved with index index-5005-08.fits.

But the astrometry package seems to be taking much time without a solution. I have provided the same hints used for the local installation. I suspect it has to do with how solve-field solves the input coordinate list. Is it possible to replicate the solve-field approach? Thank you for any help.

[aMarcireau]

Hi,

I believe astrometry.net does two things to find a solution more quickly:

• it sorts stars by increasing magnitude (brightest stars first)
• it stops searching after the first good match

Astrometry (this package) does not sort by magnitude, but you can sort the star list before calling solve if you have access to the magnitude of sources.

To stop the search after the first good match, you can use logodds_callback (example here: https://github.com/neuromorphicsystems/astrometry#return-early-if-the-best-log-odds-are-larger-than-1000).

[prajwel]

Thank you for the quick reply. I had already included both mentioned things.

In my opinion, solve-field appears to be taking 50 sources at a time and quad-matching. I partially tried the same with the astrometry solver as follows.

logging.getLogger().setLevel(logging.INFO)

solver = astrometry.Solver(
    astrometry.series_5200.index_files(
        cache_directory = "astrometry_cache",
        scales = {3,4,5},
    )
)

stars = sorted_sources[:50]

solution = solver.solve(
    stars_xs = [star[0] for star in stars],
    stars_ys = [star[1] for star in stars],
    size_hint = astrometry.SizeHint(
        lower_arcsec_per_pixel = 0.416,
        upper_arcsec_per_pixel = 0.418,
    ),
    position_hint = astrometry.PositionHint(ra_deg = ra_pnt,
                                          dec_deg = dec_pnt,
                                          radius_deg = 0.33,
    ),
    solution_parameters = astrometry.SolutionParameters(sip_order = 2, 
                                                        distractor_ratio = 0,
                                                        minimum_quad_size_fraction = 0.01,
                                                        logodds_callback=lambda logodds_list: astrometry.Action.STOP
                                                       ),
)

But this did not help. I think that while it takes 50 at a time for quad-matching, any match is checked against the total list of sources. This is why the matched indices of stars are 4, 9, 208, and 230. However, mine is only a hunch since I have not gone through the code.

[aMarcireau]

Hi @prajwel,

Sorry for the late answer. I am currently digging in the Astrometry source code to understand what their solver is doing differently. Would you mind sharing your list of stars with me, so that I can use it as a test during my investigation?

[prajwel]

Please find the list of star coordinates below. The detections are from an FUV image obtained using the AstroSat UVIT telescope.

1249.0359379734023,3556.342837088638
2904.784518931315,4215.938799401896
2417.2356951695647,4033.9408383851255
4020.5649476406957,1980.2936644194244
1614.690833214563,3017.2717434403676
2612.4920293138703,2817.5043688493092
3826.9384711164344,1426.3030103113654
1440.5734647419902,2909.225799596856
3056.7932024384063,1204.6973809943208
4142.18321858524,3472.3693177865925
1327.4898862248335,2855.348617837679
1311.8576483450333,1318.000799926427
3212.5924643943936,2888.284878861458
1092.4633177274484,805.5631104323076
1282.4456658349147,1283.7012241586656
3426.217588018623,1808.5287023968008
1118.7136161427456,873.7789329293935
2776.6053235520703,3589.7750034088463
1737.2319675334727,3354.710702820252
3278.5062870254105,2778.741328082428
3229.0477509749844,1563.2728637084704
2126.818757661776,1317.5841493239182
2498.649731117942,1362.5962729243854
2687.115842517078,3813.5526933266387
1802.010506451938,2931.683316713623
2437.2074369635075,4251.276399474872
1034.822137831218,2233.6337331479763
4280.557357013049,2734.550462099632
1866.3231431183028,996.8854976120505
2904.4544196820475,2588.930611193078
1801.0047220768358,3298.1836504720877
511.270594302231,2058.836410253311
1966.1746149660064,3199.698146198321
3685.4609576153284,1691.0580999549268
3522.73734522106,3743.25436687455
1623.4814039542134,1173.0143269210607
2078.532427879854,4336.331137978404
3168.4395957501283,3033.5249154972585
1876.1076185361553,3592.5295554633562
2678.130461476731,3800.867970057331
2730.4108031010664,4184.291748255362
3271.8900901847965,965.3421215209133
1355.8480163965678,1297.7392437063954
1094.5639629433429,1966.7568797554345
2033.1318472113726,629.689694867758
1930.3745579174993,2765.146677236126
2262.1186892362357,2577.4366720026073
1972.4466964219796,1996.0184777699376
2146.558949767421,2143.780705744067
2333.9722854716756,2377.838789730432
2277.7102102940225,2873.2472412323355
3203.3545738274343,3321.8448388260967
1258.650647634034,3117.3981915099084
2341.504973491691,1210.1069568719327
2516.662756910813,2824.8477032258907
2972.037896519286,951.0950052432469
2800.5772973629537,3093.3161311716094
1846.606078414635,1736.4982811525826
1091.9270397328144,3237.121110083393
1357.8981637359138,1406.7744620183362
2340.669302268481,861.3538909530163
1165.350107236046,1952.9055073961392
2153.3518894068598,1222.1256250563401
672.5222236522892,2837.878291592293
3281.26099257424,976.8087675698579
813.1773981295519,3576.5837463079297
1665.4672842922357,2554.5897233520272
3331.626292181266,3940.612271019419
2299.7562643336682,3172.722677977063
3449.245055646054,3019.7001155082858
2973.38815036789,1797.7059798405915
1022.5220545403781,2730.8380066546406
743.8652981921483,1740.281047424187
4082.667560301175,2984.6793383821037
2947.2316710728787,4178.72299523797
1607.049591772274,3580.9974701400874
1331.8173202731346,1852.56471830457
1643.8020140041958,3819.811012150187
3067.4770231929224,3920.984279828466
1385.7780533322964,1283.1483080872729
3703.674510702543,2071.922298968308
1643.5082425214414,3819.76586883477
3096.2076758093,3254.881555275546
936.7920626879178,794.1977559943319
2891.951001402589,3591.0055005773506
3319.7267895643936,3029.494524577007
3108.356748500565,4062.6156011381677
3736.2171803520264,2580.018197042115
2438.7608933405054,3284.296822550912
1578.00780746738,2873.89061784107
2460.310697248418,735.1001149413111
1011.4818035648282,3315.9934246157068
2689.1021976739953,660.3457411464765
2687.9531618347964,3631.839653897643
1058.1275452903967,3083.25505785098
1837.7540646740747,3795.517292374611
4195.444868052262,2858.0567754540616
2147.699991005858,2921.312859568679
1661.3061491761562,2389.84550758179
1975.3491966803726,2579.0235615955025
2925.609433294442,3760.9243760338404
991.6447537590261,1736.1842874708393
596.3847399626189,2899.242212231364
2572.922417028569,3148.96798021108
1202.9546009342819,2183.03600158086
3399.867342363576,3740.1483303966784
1552.084076361392,3500.23083609646
2473.1900374714546,2115.71871921459
819.3225511359637,1689.757161591397
3412.132816118448,3309.122995576647
1098.5885931797193,1664.901803646012
2062.8336382208918,1727.888380485879
3655.3711322245376,2375.4066077511225
1054.736123930808,3006.0111300066515
1571.675934003225,4097.895119009508
3676.9830415194615,3287.2988137014468
1508.542406801225,3019.9072725925566
3808.0710841189866,2957.3649048345724
2291.8105740309497,3943.058372068256
1616.9537360058503,3919.634560475895
4187.811584276098,2838.704068282026
1103.3903491468132,2730.5690385290795
961.8763717986383,2493.8274048188896
1172.1622323370727,1313.6724227583147
2439.981210795216,1547.0985395846378
1978.772238542741,1974.0628450430656
1614.0839558695884,1240.7714780796907
4344.6074241982815,2479.536032355109
1812.503732450196,2960.3827963725143
3022.0017850761183,2362.323005859566
1282.2635689665135,1958.0608377840667
3768.0891757743757,2016.3300002217902
3940.1070910932694,3426.237070416937
2195.0134047207634,1570.8488010007682
2237.844953155282,4057.943980056299
3764.2867767056273,2303.167774423024
4260.771668064927,2929.3817741186062
3349.2367269367796,1020.0436639789456
3415.211918625248,1660.9817770393547
1935.778093667702,2591.102672495771
2430.326386510017,1644.4861548744423
3099.489555588844,1050.4831747580226
3763.903199082098,1116.5378428934303
2364.2703967278785,3912.506327771856
3108.567405612556,3405.409745270876
3744.5694334049495,1525.7221829925184
3108.265978475273,659.7015496754726
3678.0749032111153,3418.356254977604
3867.4696776144183,1003.2878073359468
2100.687872775736,4249.713071363531
3061.0328261448894,2651.384549813812
1749.5811744066623,2391.836701898808
2532.5741571401636,3426.638193394974
3946.40176622876,2663.7776605493855
2704.166847831781,1104.7531932625682
1171.2410765999402,3115.120834109872
1176.4755072099042,1978.3214365431302
2949.8820209497044,3372.2926836129814
2635.3518681702767,2918.88890464211
553.2421644982157,1658.2903338536883
689.508362369821,1506.5213912217348
2885.9155480708623,3603.118546311456
772.0975083145433,3170.9410243166676
2692.347201221916,3599.186313766261
2600.63385225327,2978.155649470924
2760.600864891266,3259.4344468415497
1226.61321591861,3813.105521589276
3071.6538987255917,2699.5705104106282
2021.1151396920416,1118.6984650164632
3967.05383868566,2756.1813230236826
1320.9585796422111,1575.787298726635
2226.114135474068,2369.277576289822
2599.6126691450204,1680.0461975410633
2667.6415052068683,4034.178864801299
3194.5185891001797,2863.6260332679576
2482.8078490420635,2551.9969286721634
1868.3690070231773,657.8909796272696
1106.749800816822,969.7264464989571
4101.155067147113,3038.7260817195906
3259.517278847022,1151.550118526319
2656.263697859305,3776.845241128075
3652.752466310236,1294.0098806722542
2576.371783547132,619.8073056790222
1048.2064016687034,2029.281089884922
2604.500637851027,1725.2897751424755
682.04639281032,3038.418662814478
2045.1972290752615,2558.8480622474317
3200.726159817141,3514.1007650422175
540.5870293080777,2143.1413601473037
2358.676480622296,604.9998120803634
1386.3535033370392,1204.9113935803985
1195.3924401272416,3979.8700373164857
1527.9893052861544,570.5544881570705
1646.1553789825064,3836.089037542008
3673.990402542719,1475.8883693475086
2761.456949071901,3293.520284452599
1513.1471416080535,1146.2225678663121
3389.1955123435173,995.0171006664931
3231.7678690796333,987.3972367897252
3077.886929776497,3273.1114844896424
2673.8096194265886,3661.423105433173
3559.190203209018,3771.8920598137274
2885.7864776320284,3284.6764730549307
2352.8084714174706,3699.6786713494657
1897.874047448667,3860.221420893699
4014.519665511381,1384.5892893649598
2435.279494149729,2557.973414399773
2674.170228997126,1318.2310798361566
2958.8157384905207,3043.3389303983595
2994.2896592150855,2876.828330528243
3627.4489678448926,901.941112366983
2016.3273210718178,2756.0810087499767
2141.8001269667925,3472.7809561771596
2549.4739087797498,2906.3625362395205
1307.423666956217,3457.3555150620637
402.4981734640036,2230.816259336342
1101.6988886285137,1646.1002944485047
2958.8223000027256,2619.26822562893
2098.2097599558588,683.7072480718887
4135.136184813181,1562.9406376328927
3196.6017714077966,3127.2988102368645
2660.6345521300045,3853.563909999319
3516.9910427123177,2994.071101216528
2921.448170166068,387.46468956458637
3222.8258260049633,3091.301734978586
4054.7990311110325,1432.2206907047564
2331.1638354479455,1302.4002094554974
1223.5997247224243,3983.7586703429793
3578.171804962619,1904.5679841233004
1513.0507955312746,2976.7305417335556
3108.2578262156194,2038.7949293922056
3597.7127122000243,3786.6130270153394
2214.3425721168755,1016.039447229016
2572.549551150622,2980.388537873875
2600.676425768898,2920.3422593218693
2375.270080424454,4165.109086181978
1615.076373389533,3403.174870029817
1639.3933276255932,3622.3720080157645
701.0194016689383,3331.901611441927
2606.0608452892625,628.3011090937455
1107.2360107643672,2182.67950744249
816.2411872806427,1673.4284240005009
2310.87659342822,3916.161628545245
4079.9302367246373,1677.4755143113405
592.8892612121568,1917.707259510901
1711.237223238647,2716.3811607924586
429.13540954149323,2809.417228315836
2157.7556108146455,2341.783778882727
927.4218961360979,3663.38379925805
751.4395163968685,3335.664858937225
1332.3262766033897,808.1868251962333
2998.0869464129137,3643.8243029480964
1704.1480135077543,3798.4038696510975
2793.889554723997,2536.924462505973
3150.7697763846863,1420.3930723434755
767.5238899817158,2403.100874976611
2796.1145168345156,2543.918432107674
1924.0261180570103,1618.4365391142076
2370.1875567738057,2527.162763374013
3778.5854356011578,3482.416852000576

[aMarcireau]

@prajwel I upgraded astrometry to try and align its behaviour with Astrometry.net. I tested your star list with this new version (see https://github.com/neuromorphicsystems/astrometry/blob/main/test/test_astrosat.py) and got results that are compatible with but slightly different from solve-field.

I do not fully understand why the results are different (perhaps a difference in the stars extraction method or order?), but I believe that the new version is a step in the right direction.

[prajwel]

I tested the new version, and it works. Thank you! I will close this issue now.