Not really an issue, collaboration

[rlancaste]

Hi There, I am Rob Lancaster and I have been involved with KStars and INDI since about 2016. For years, we used astrometry.net with KStars and added ASTAP support, but we always ran into issues as you found. We always had huge issues with configuration files, users having multiple python versions, misconfigurations etc. And of course astrometry.net doesn't natively run on windows and needs a compatibility layer So in 2020, around the time of the start of the pandemic, I developed StellarSolver, a cross platform Library that a cross platform Qt program can use to plate solve in a variety of ways. As a part of StellarSolver, I did dive into the astrometry.net code and made it work on Windows natively along with making it work as an internal library needing no external configuration files. So it does have a built-in stripped down version of astrometry.net in there. But in addition to this internal solver, it can also use ASTAP, local astrometry.net, Windows ANSVR, Windows Cygwin, and online astrometry.net. Its real strength however is in how it can use most of these solving methods in a parallelized manner and can thus blind solve many jpegs with them in less than 10 seconds ( or less than 1 second on my 16 core I9 processor). The whole project with all solving methods works equally well on Windows, Mac, and Linux. I also have had many conversations with Han who created ASTAP.

Your solver sounds very interesting and I would be interested in supporting that one as well if you think it is ready and able to work on all platforms. I will download it and give it a shot.

[Jusas]

Yeah, it should actually throw an error that says it in stdout, but I'll need to double check that. If it doesn't, I'll fix it. In any case fortunately the solver isn't yet in widespread use so it shouldn't be a big issue.

[rlancaste]

Sure, not yet, I think most KStars users are using one of the other options already, so the addition of this won't cause them to switch unless we really promote it. They might see that in the menu and say "Hm, what is that?" Now that could change when we do the next kstars release which shouldn't be too far away. It might be in a few days or maybe a week?

[Jusas]

Yeah, I expect people will be more interested in it when it becomes a viable option being actually integrated to a popular application. StellarSolver itself already does a great job so I don't expect a huge following but then again KStars is just one use case for Watney. In any case, I'll be focusing on the polish for the next few days and will probably push out a new minor release with small issues like these ironed out.

[rlancaste]

Yep, but half the point of my making stellarsolver was to support multiple solving options and to make them easy for users to handle. It was crazy with all the different configuration files, programs, options, and settings. They all had different options and settings and everybody had trouble with this that or the other thing on this solver or that solver. And of course that was all tangled up into kstars code making code spaghetti. So my goal was to move it all into an external library, streamline it, perfect it, try to avoid any config files where settings could be incorrect, and make a one stop shop for options. Then to customize those options to try to make sets that are better at certain tasks, which are my options profiles. So the complicated part gets wrapped up inside StellarSolver and it provides a (fairly) simple yet powerful external interface to try to handle it all.

[rlancaste]

And of course, as much as possible, at least in the case of my internal library build of astrometry, to avoid writing any files to the hard drive to save the PI and its memory card.

And to make Astrometry.net natively run on windows of course.

[rlancaste]

And on Mac to get rid of our stupid python problems with all the users having different pythons with different permissions and not having them set up right. . .

[rlancaste]

but yes, I digress. My point was that your Watney solver will provide another nice option for users. And including it will let all of us us play around with all of the solving methods in my tester and we can make them all better.

I now have it to the point that I can blind solve many images with a couple of solving methods in Stellarsolver on my system in less than a second. That's largely a result of just playing around in the tester to see what I could improve over the last couple of years.

So adding another option is good in my opinion. That's how we get better.

[Jusas]

I agree :) and I'm happy to provide another option. I can understand that it's a difficult and painstaking job to cover all the different solver options and making things just work, and I can say that since this is all new to me I'm certainly not making it very easy for you coming up with new options and changing things rapidly... but since this is something new, if there's a time to do it it's now. So thank you for all the effort.

And yeah, the more solver implementations we get the better. I can see each of us have somewhat different implementations, motivated by different ideas. I always favored simplicity above all, and making it easy to use. I intentionally didn't really look into other implementations when I started (except for ASTAP for the main algorithm) to see and find out on my own how to actually make it work fast. I still haven't looked into using SIMD for example, or even GPU acceleration but that's something interested I might at some point experiment with. So far I've been able to squeeze the milliseconds out without those and besides, a lot of the hardware that runs the solver is lower tier and in most cases the few milliseconds wouldn't even matter too much. And of course portability is something I'm not willing to sacrifice at any point, this must be able to run well on all platforms. Blind solves remain the most challenging thing to optimize and I think that there are still some eureka moments to come with that to further speed it up.

Anyway, fun stuff. And I've gotta say you've made one fantastic job with StellarSolver!

[rlancaste]

So, one thing I would ask is if you can let me know when you need to change something that will require the code to be changed or updated, that would be great. As of now, I think it's running pretty well in Stellarsolver, so let's try to keep it working well. You could even make a pull request if you like. In terms of outward facing options for the GUI, however, I would like to try to not add too much more. Plate solving is already pretty complex for users, I would like to try to simplify things for them as much as possible. If we can automatically configure some settings, that is usually better than just providing options for them in the GUI.

[Jusas]

Yes, definitely. And I will try to keep it backwards compatible whenever possible. And I will probably start making pull requests once we get this pushed out the door.

[han-k59]

Hello Jussi, Hello Rob,

Jussi, congratulations with your solver! Your indexed quads works faster then astrometry.net indexes in FITS tables :)

You both have been busy. I just popped in and read only parts of the conversion.

Rob, SEP is an excellent star detector, but I lost interest in xyls tables since the ASTAP star detection is pretty good and xyls interface poluted the code.

Han author ASTAP.

[Jusas]

Hello Jussi, Hello Rob,

Jussi, congratulations with your solver! Your indexed quads works faster then astrometry.net indexes in FITS tables :)

You both have been busy. I just popped in and read only parts of the conversion.

Rob, SEP is an excellent star detector, but I lost interest in xyls tables since the ASTAP star detection is pretty good and xyls interface poluted the code.

Han author ASTAP.

Thank you Han, I appreciate it! :)

[rlancaste]

Thanks Han, yes we have both been busy

[han-k59]

Jussi,

I understand that the Gaia database is organised on celestial location. Have you ever considered to sort the database on quad hash code? I assume this will be only beneficial for blind solving.

I did some more testing today with Watney and the star detection seems to work well. I doubt if it is required to go to an other star detection library.

[Jusas]

Jussi,

I understand that the Gaia database is organised on celestial location. Have you ever considered to sort the database on quad hash code? I assume this will be only beneficial for blind solving.

This is a good point - sorting would increase speed somewhat. That's something I definitely need to test, it might indeed be quite significant since we could drop a lot of unnecessary calculations. Why didn't I think of that before 😄

[han-k59]

It may not be efficient for nearby solving which are the bulk of solve requests.

[Jusas]

Guys, I'm going to close this issue since this is now more or less a chat. Open up new ones if you run into issues.