solve_from_image wont solve

[windell747747]

Hi, I have a picture that i've taken and it solves using astrometry.net, but it wont solve using tetra3. The code works for the test images, however, when I substitute the image that I took, it doesnt solve.

Im wondering if there there is something that I need to do to the image for the solver to work?

I've changed the match_threshold to 0.1 and the matched_stars to 6, and the match_radius to 0.1. The camera horizontal FOV is 11.6 deg.

[lcandell]

I was able to get your image to solve, but I needed to change 2 default parameters. The default max_area=100, and I noticed that your 2 brightest stars exceeded this threshold (with the default sigma=2). I made the max_area=200.

Also, I would suggest making the pattern_checking_stars parameter higher than the default of 8. I found that I need to make it more like 15 to get robust results. In the case of your image, it appears that it wound up solving on stars 1, 2, 5, and 11 in brightness order. Making this parameter higher will slow down the solver, but at a level of 8 it is very unforgiving. With the setting of 15, it actually solves even without the two brightest stars.

I hope this is helpful.

[windell747747]

Thank you very much for helping me with this! This is incredibly helpful. Would you be able to give me an explanation of these parameters? I took a look at the documentation and I dont see how I would be able to deduce changing these parameters. Is the max_area, the max area of a centroided source? Also, patter_checking_stars is the minimum number of stars assumed to be in the image?

Thank you for your help!

On Mon, Sep 25, 2023 at 9:08 AM lcandell @.***> wrote:

I was able to get your image to solve, but I needed to change 2 default parameters. The default max_area=100, and I noticed that your 2 brightest stars exceeded this threshold (with the default sigma=2). I made the max_area=200.

Also, I would suggest making the pattern_checking_stars parameter higher than the default of 8. I found that I need to make it more like 15 to get robust results. In the case of your image, it appears that it wound up solving on stars 1, 2, 5, and 11 in brightness order. Making this parameter higher will slow down the solver, but at a level of 8 it is very unforgiving. With the setting of 15, it actually solves even without the two brightest stars.

I hope this is helpful.

— Reply to this email directly, view it on GitHub https://github.com/esa/tetra3/issues/13#issuecomment-1734314562, or unsubscribe https://github.com/notifications/unsubscribe-auth/A74GBIX2KRQNDBMMMKQNF7DX4HJEJANCNFSM6AAAAAA4UDLWFE . You are receiving this because you authored the thread.Message ID: @.***>

[lcandell]

max_area: After the centroid algorithm subtracts a background estimate and applies a threshold to the image, it collects a set of connected components (contiguous blocks of pixels that all exceed the threshold). The area is simply the number of pixels included in the centroid. For you image, the brightest stars (mag3.6) lead to a large area. max_area removes any centroids found that exceed this area size.

pattern_checking_stars: The solver algorithm attempts to find groups of 4 stars (quads) in its database that have a similar shape to subsets of 4 centroids in your image. Looking through the code, the pattern_checking_stars parameter limits the number of centroids considered in these 4 star groupings to the brightest set of the specified length. The parameter defaults to the 8 brightest in the code, which is probably sufficient for many scenarios. Making it higher checks more centroid combinations until a match is found.

Properly setting any of these parameters will depend a fair bit on how you plan to use the solver in practice, as well as the details of your hardware. I've been impressed that, for a given setup, it can be configured to be quite robust and fast.

[windell747747]

Thank you very much for such a quick and thorough response.

Since it rains a lot where I am, I have a setup in my office that has Stellerium on a matte surface monitor. I adjust the FOV on the monitor to ensure that the FOV of the image matches the catalog. Unfortunately, to simulate the varying brightnesses across the image, Stellerium can't adjust the pixel brightnesses on the monitor to the proper ratios and instead makes them have a larger diameter to compensate. I think this is why the areas are so large.

From your response it seems that the larger max_area and that larger pattern_checking_stars would increase the robustness of the solver? It seems that the max_area is to reject false positives in the image that aren't stars? So of course, too large of a max_area value can be detrimental.

How did you look up the magnitude of the brightest stars in my image? I know I can do it manually, but I'm wondering if there is output somewhere that I'm not seeing that might be able to help me debug in the future.

Thanks again! I work at the Canada-France Hawaii Telescope for my day job and I owe you a beer and one of our calendars! Please give me an address to send it too!

On Mon, Sep 25, 2023 at 11:00 AM lcandell @.***> wrote:

max_area: After the centroid algorithm subtracts a background estimate and applies a threshold to the image, it collects a set of connected components (contiguous blocks of pixels that all exceed the threshold). The area is simply the number of pixels included in the centroid. For you image, the brightest stars (mag3.6) lead to a large area. max_area removes any centroids found that exceed this area size.

pattern_checking_stars: The solver algorithm attempts to find groups of 4 stars (quads) in its database that have a similar shape to subsets of 4 centroids in your image. Looking through the code, the pattern_checking_stars parameter limits the number of centroids considered in these 4 star groupings to the brightest set of the specified length. The parameter defaults to the 8 brightest in the code, which is probably sufficient for many scenarios. Making it higher checks more centroid combinations until a match is found.

Properly setting any of these parameters will depend a fair bit on how you plan to use the solver in practice, as well as the details of your hardware. I've been impressed that, for a given setup, it can be configured to be quite robust and fast.

— Reply to this email directly, view it on GitHub https://github.com/esa/tetra3/issues/13#issuecomment-1734456037, or unsubscribe https://github.com/notifications/unsubscribe-auth/A74GBISPMPVSXGCKA3IKV4TX4HWIPANCNFSM6AAAAAA4UDLWFE . You are receiving this because you authored the thread.Message ID: @.***>

[lcandell]

That certainly a unique application for a star tracker! It does explain some of the artifacts of your image, too. I do not believe that Tetra3 has a provision for reporting visual magnitude, though it can convert centroids to an RA/Dec that can be looked up in a catalog. Best of luck at CFH!

[gustavmpettersson]

Hi,

Big thanks to @lcandell for helping here. I have a hard time finding gaps to work on tetra3. Your explanations are all correct. Just for context, I picked the default of 8 pattern checking stars as this will fail in only a few seconds, which is reasonable to me. The default catalogue is a little bit sparse to not make it silly large, hence some patterns are missing. If I understand your description correctly that this is a photograph of a screen, the algorithm is likely searching in the "wrong order" because it is hard to sort the stars by brightness.

For the last point, note that you can pass return_matches=True, then you will get back three extra keys in the results with information about the matched stars in the image. matched_stars gives the RA, Dec, and Mag (taken from the database, not calculated from the image), matched_centroids gives the corresponding pixel coordinates, and matched_catID gives the ID in the original catalogue, which for the default is the HIP number.

Cheers, Gustav

[windell747747]

Thank you, Gustav! Is there a way to disable needing to get the relative brightnesses correct and instead just use positions of the stars instead? Say I had a binary mask for the stars for the same image? This way it does not assume that the relative brightnesses are correct and the solver isn't led down the wrong path?

On Mon, Sep 25, 2023 at 1:10 PM Gustav Pettersson @.***> wrote:

Hi,

Big thanks to @lcandell https://github.com/lcandell for helping here. I have a hard time finding gaps to work on tetra3. Your explanations are all correct. Just for context, I picked the default of 8 pattern checking stars as this will fail in only a few seconds, which is reasonable to me. The default catalogue is a little bit sparse to not make it silly large, hence some patterns are missing. If I understand your description correctly that this is a photograph of a screen, the algorithm is likely searching in the "wrong order" because it is hard to sort the stars by brightness.

For the last point, note that you can pass return_matches=True, then you will get back three extra keys in the results with information about the matched stars in the image. matched_stars gives the RA, Dec, and Mag, matched_centroids gives the corresponding pixel coordinates, and matched_catID gives the ID in the original catalogue, which for the default is the HIP number.

Cheers, Gustav

— Reply to this email directly, view it on GitHub https://github.com/esa/tetra3/issues/13#issuecomment-1734585715, or unsubscribe https://github.com/notifications/unsubscribe-auth/A74GBIQHXQ6E3MA36UGRJW3X4IFPVANCNFSM6AAAAAA4UDLWFE . You are receiving this because you authored the thread.Message ID: @.***>

[windell747747]

Thank you, Gustav! Is there a way to disable needing to get the relative brightnesses correct and instead just use positions of the stars instead? Say I had a binary mask for the stars for the same image? This way it does not assume that the relative brightnesses are correct and the solver isn't led down the wrong path?

On Mon, Sep 25, 2023 at 1:10 PM Gustav Pettersson @.***> wrote:

Hi,

Big thanks to @lcandell for helping here. I have a hard time finding gaps to work on tetra3. Your explanations are all correct. Just for context, I picked the default of 8 pattern checking stars as this will fail in only a few seconds, which is reasonable to me. The default catalogue is a little bit sparse to not make it silly large, hence some patterns are missing. If I understand your description correctly that this is a photograph of a screen, the algorithm is likely searching in the "wrong order" because it is hard to sort the stars by brightness.

For the last point, note that you can pass return_matches=True, then you will get back three extra keys in the results with information about the matched stars in the image. matched_stars gives the RA, Dec, and Mag, matched_centroids gives the corresponding pixel coordinates, and matched_catID gives the ID in the original catalogue, which for the default is the HIP number.

Cheers, Gustav

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[gustavmpettersson]

Unfortunately no, the catalogue is built from bright stars, so the solver searches them in that order too. If you can generate a brightness-sorted list of all pixel coordinates you can use solve_from_centroids instead. Is this application just experimentation? Even a screenshot may solve.

[windell747747]

Right now I'm using Stellarium and taking images of the screen using the camera which tries to represent the relative brightnesses, but doesn't do well because the pixels don't have enough dynamic range. So instead to compensate, the diameter of the star is increased.

The brightness of the relative brightness of the star is based on the total encircled flux is that correct?

So there is no way to configure the solver to be more robust so its solution on only the pattern of the stars and not include brightness as a constraint? Naturally, the camera will only pick up the brighter stars, so I think the image will always only include bright stars. This would also make the solver robust to camera saturation, although I have an algorithm that takes care of the saturation. If I have the pattern checking stars high would it work, I just want to make sure to understand all the knobs.

On Tue, Sep 26, 2023 at 5:00 AM Gustav Pettersson @.***> wrote:

Unfortunately no, the catalogue is built from bright stars, so the solver needs to be able to guess which those are. If you can generate a brightness-sorted list of all pixel coordinates you can use solve_from_centroids instead. Is this application just experimentation? Even a screenshot may solve.

— Reply to this email directly, view it on GitHub https://github.com/esa/tetra3/issues/13#issuecomment-1735723767, or unsubscribe https://github.com/notifications/unsubscribe-auth/A74GBIUQDRJBQCM5MGGF3GTX4LUZZANCNFSM6AAAAAA4UDLWFE . You are receiving this because you authored the thread.Message ID: @.***>

[lcandell]

I got a chance to look at the detected stars in the image you provided. The brightest ones are listed below, sorted by the order that Tetra3 calculates the centroid Mass: RA DEC Mag Centroid Mass Centroid Area 10.176 -12.355 3.61 21844.96 189 10.435 -16.837 3.82 19085.91 168 10.085 -13.064 4.6 15450.478 136 10.168 -12.817 5.3 13420.253 123 10.119 -17.142 5.58 11885.257 105 10.067 -18.101 5.85 10621.33 102 10.321 -12.528 6.03 9392.013 103 10.269 -11.203 6.07 9028.225 103 10.143 -15.612 6.25 7981.336 92 10.166 -12.095 6.23 7969.7905 97 10.37 -19.867 6.13 7687.7847 91 10.21 -19.154 6.46 7500.164 79 10.352 -17.985 6.51 5956.645 74 10.279 -20.67 6.58 5190.668 75 10.115 -17.444 6.72 4421.47 86 10.276 -19.308 6.71 4387.0684 66 10.189 -18.958 6.8 4241.0273 59 10.477 -19.525 6.71 4158.7285 73 10.594 -18.569 6.48 3796.272 48 10.042 -13.296 6.99 2928.4363 69

As you can see, the centroiding algorithm does a great job of finding the stars in the image, and it even orders them with very good correspondence to the actual visual magnitude. This run uses the default centroid parameters, but changes the max_area to 200 instead of 100. (You can see that most of the brightest stars exceed the default area threshold)

Here’s the solution that the solver computes:

'RA:153.5087', 'Dec:-16.8730', 'Roll:288.3810', 'FOV:11.5883', 'distortion:-0.0209', 'RMSE:103.7019', 'T_solve:71.5631'

[gustavmpettersson]

I'm closing this issue as the original image has been solved. Feel free to keep discussing here.

[windell747747]

Thank you very much!

For my reference, would you mind sending me your code generate the list? I modified my code using your suggestions and it looks like it solves now. However, since you are a pro, I would like to see how you've setup your arguments and debugging images. I want to really understand how to ensure a high level of reliability and robustness in the code and how I can debug what is going wrong when it isn't solving.

Thank you, Windell

On Wed, Sep 27, 2023 at 2:34 PM lcandell @.***> wrote:

I got a chance to look at the detected stars in the image you provided. The brightest ones are listed below, sorted by the order that Tetra3 calculates the centroid Mass: RA DEC Mag Centroid Mass Centroid Area 10.176 -12.355 3.61 21844.96 189 10.435 -16.837 3.82 19085.91 168 10.085 -13.064 4.6 15450.478 136 10.168 -12.817 5.3 13420.253 123 10.119 -17.142 5.58 11885.257 105 10.067 -18.101 7.09 10621.33 102 10.321 -12.528 6.03 9392.013 103 10.269 -11.203 6.07 9028.225 103 10.143 -15.612 6.25 7981.336 92 10.166 -12.095 6.23 7969.7905 97 10.37 -19.867 6.13 7687.7847 91 10.21 -19.154 6.46 7500.164 79 10.352 -17.985 6.51 5956.645 74 10.279 -20.67 6.58 5190.668 75 10.115 -17.444 6.72 4421.47 86 10.276 -19.308 6.71 4387.0684 66 10.189 -18.958 6.8 4241.0273 59 10.477 -19.525 6.71 4158.7285 73 10.594 -18.569 6.48 3796.272 48 10.042 -13.296 6.99 2928.4363 69

As you can see, the centroiding algorithm does a great job of finding the stars in the image, and it even orders them with very good correspondence to the actual visual magnitude. This run uses the default centroid parameters, but changes the max_area to 200 instead of 100. (You can see that most of the brightest stars exceed the default area threshold)

Here’s the solution that the solver computes:

'RA:153.5087', 'Dec:-16.8730', 'Roll:288.3810', 'FOV:11.5883', 'distortion:-0.0209', 'RMSE:103.7019', 'T_solve:71.5631'

Sent from Mailhttps://go.microsoft.com/fwlink/?LinkId=550986 for Windows

From: @.> Sent: Tuesday, September 26, 2023 4:31 PM To: @.> Cc: @.>; @.> Subject: Re: [esa/tetra3] solve_from_image wont solve (Issue #13)

Right now I'm using Stellarium and taking images of the screen using the camera which tries to represent the relative brightnesses, but doesn't do well because the pixels don't have enough dynamic range. So instead to compensate, the diameter of the star is increased.

The brightness of the relative brightness of the star is based on the total encircled flux is that correct?

So there is no way to configure the solver to be more robust so its solution on only the pattern of the stars and not include brightness as a constraint? Naturally, the camera will only pick up the brighter stars, so I think the image will always only include bright stars. This would also make the solver robust to camera saturation, although I have an algorithm that takes care of the saturation. If I have the pattern checking stars high would it work, I just want to make sure to understand all the knobs.

On Tue, Sep 26, 2023 at 5:00 AM Gustav Pettersson @.***> wrote:

Unfortunately no, the catalogue is built from bright stars, so the solver needs to be able to guess which those are. If you can generate a brightness-sorted list of all pixel coordinates you can use solve_from_centroids instead. Is this application just experimentation? Even a screenshot may solve.

— Reply to this email directly, view it on GitHub https://github.com/esa/tetra3/issues/13#issuecomment-1735723767, or unsubscribe < https://github.com/notifications/unsubscribe-auth/A74GBIUQDRJBQCM5MGGF3GTX4LUZZANCNFSM6AAAAAA4UDLWFE>

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub< https://github.com/esa/tetra3/issues/13#issuecomment-1736249810>, or unsubscribe< https://github.com/notifications/unsubscribe-auth/AP5J4LVT6UEEPPO6VJYYYXTX4M3Q5ANCNFSM6AAAAAA4UDLWFE>.

You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/esa/tetra3/issues/13#issuecomment-1738282314, or unsubscribe https://github.com/notifications/unsubscribe-auth/A74GBIS3CQ7RN52CK34DKLDX4TAZ5ANCNFSM6AAAAAA4UDLWFE . You are receiving this because you authored the thread.Message ID: @.***>

[gustavmpettersson]

Hi Windell,

I don't have Icandell's script, but can hopefully help with the understanding.

Firstly, I imagine you run your images now with e.g.: solved = t3.solve_from_image(img, max_area=1000, pattern_checking_stars=15)

There are two key steps in tetra3, centroiding and solving. Firstly, you can get a useful view of the centroiding results by passing return_images=True. (This is passed as an argument to get_centroids_from_image.) You will now get a second return, where you can look especially at final_centroids:

solved = t3.solve_from_image(img, max_area=1000, pattern_checking_stars=15,
            return_images=True)

        if isinstance(solved, tuple):
            centroid_images = solved[1]
            solved = solved[0]

            centroid_images['final_centroids'].show()

This gives you green circles for the extracted centroids, red circles for rejected centroids (originally, your four brightest were rejected due to size).

Next, to get some insight into a solution (this only works if the solve is successful), there are more returns you can ask for. Add return_matches=True and return_visual=True`, to get additional data about the matches and a visualisation of the solution respectively:

solved = t3.solve_from_image(img, max_area=1000, pattern_checking_stars=15,
            return_images=True, return_matches=True, return_visual=True)

        if isinstance(solved, tuple):
            centroid_images = solved[1]
            solved = solved[0]
            centroid_images['final_centroids'].show()

        if 'visual' in solved:
            solved['visual'].show()

        print('Solution:', solved)

In the solution, matched_centroids and matched_stars list the pixel coordinates and star data (RA, Dec, Mag) for each match. There is also matched_catID giving the catalogue ID for newer databases.

If you want to really look into the details of centroiding, you can call get_centroids_from_image yourself, especially with return_moments=True, you will get the statistics (area, sum, second order moments) for each centroid.

[windell747747]

Thank you very much, Gustav! I apologize for the late response. I understand now and I apologize for not reading the documentation more thoroughly. All of my questions would've probably been answered.

I've been working a lot with the previous version of tetra3 since March of 2023 in one application and more recently I've upgraded to the latest version. I read that there was a long list of revisions made. Is there a description of the changes and effects somewhere?

The old code has been working fine for me and I would like to evaluate the effect of moving over to the new version. In my code, I compensate for distortion a little more generally by correcting for FOV, radial distortion, tangential distortion, and distortion center instead of correcting using pincushion simplification so I would need to migrate my code over to the new code if I implement it the same way.

Thank you very much again for all you're help! windell

On Sat, Oct 7, 2023 at 5:40 AM Gustav Pettersson @.***> wrote:

Hi Windell,

I don't have Icandell's script, but can hopefully help with the understanding.

Firstly, I imagine you run your images now with e.g.: solved = t3.solve_from_image(img, max_area=1000, pattern_checking_stars=15)

There are two key steps in tetra3, centroiding and solving. Firstly, you can get a useful view of the centroiding results by passing return_images=True. (This is passed as an argument to get_centroids_from_image.) You will now get a second return, where you can look especially at final_centroids:

solved = t3.solve_from_image(img, max_area=1000, pattern_checking_stars=15, return_images=True)

    if isinstance(solved, tuple):
        centroid_images = solved[1]
        solved = solved[0]

        centroid_images['final_centroids'].show()

This gives you green circles for the extracted centroids, red circles for rejected centroids (originally, your four brightest were rejected due to size).

Next, to get some insight into a solution (this only works if the solve is successful), there are more returns you can ask for. Add return_matches=True and return_visual=True`, to get additional data about the matches and a visualisation of the solution respectively:

solved = t3.solve_from_image(img, max_area=1000, pattern_checking_stars=15, return_images=True, return_matches=True, return_visual=True)

    if isinstance(solved, tuple):
        centroid_images = solved[1]
        solved = solved[0]
        centroid_images['final_centroids'].show()

    if 'visual' in solved:
        solved['visual'].show()

    print('Solution:', solved)

In the solution, matched_centroids and matched_stars list the pixel coordinates and star data (RA, Dec, Mag) for each match. There is also matched_catID giving the catalogue ID for newer databases.

If you want to really look into the details of centroiding, you can call get_centroids_from_image yourself, especially with return_moments=True, you will get the statistics (area, sum, second order moments) for each centroid.

— Reply to this email directly, view it on GitHub https://github.com/esa/tetra3/issues/13#issuecomment-1751741138, or unsubscribe https://github.com/notifications/unsubscribe-auth/A74GBITTYFWE4ILKDP6R4CDX6FZWLAVCNFSM6AAAAAA4UDLWFGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJRG42DCMJTHA . You are receiving this because you authored the thread.Message ID: @.***>