Improve LEISA camera model

[rbeyer]

Description
Allow cubes resulting from leisa2isis to be more cartographically usable.

The New Horizons LEISA camera model, as implemented, doesn’t really allow robust cartographic work, maybe it isn’t just the camera model? Application of qtie / qnet / jigsaw to LEISA cubes fails. When it was implemented, we maybe just didn’t do enough design iteration or supply high-enough quality test data. Not sure. The problem may be that the unique geometric situation of LEISA isn’t adequately described by the SPICE data, or isn’t taken into account by the ISIS software.

[jessemapel]

Can you provide a more detailed explanation of what you want to do and where things are failing in qtie/qnet/jigsaw.

[jessemapel]

Also, is this all LEISA images, only specific observations, only specific phases of the mission, etc.?

[kberryUSGS]

I know that thruster firing during image acquisition caused some difficulty with using jigsaw, etc for LEISA images due to assumptions ISIS was making about smooth trajectories -- is this a related issue, something else, or a broader problem?

[rbeyer]

My apologies, I raced to get this issue put up and it is vague. I am working on providing a more detailed test case and description.

[rbeyer]

Hmm, maybe this is more about qnet/qtie than it is about the camera model:

When a LEISA image is read in via leisa2isis, it is a 256 band cube, with each band being one of the 256 spectral channels recorded by the instrument. However, not all channels have 'good' information, for example here is what band 1 of the LSB_029916933 image looks like:

If you didn't know any better, you might think this image was bad, but there are 256 of these, and some of them are good. The other thing is that due to the way that the LEISA captures pixels, each sample and line does not correspond to the same 'spot' on the target, and so if you were to take some bands that do have Pluto recognizable, you'll notice that they don't line up. Here's an example of putting bands 77, 92, and 137 in R, G, and B:

However, I think the camera model actually is okay, because if I spiceinit this file, and then record the reported lat/lon coordinates of a known feature in each of the different bands, the reported lat and lon aren't more than a degree different. Also, you'll notice that Pluto looks 'heart-shaped' in the bands above, and that's because there was a 'thruster-firing' in the middle of this observation. Again, I think the camera model is doing okay, because if I pull out one band where I can 'see' Pluto, and run grid, the graticule is deformed as I'd expect it:

So I have a cube with 'good' SPICE data, and now I want to bundle adjust it. Since I have only this one image, and a controlled LORRI mosaic, I'll use qtie (but I could also use qnet if I had more LEISA images, I suppose).

When you open qtie, with the LEISA cube and a controlled LORRI mosaic, you can use the qtie band selector to pick a band in the LEISA image (like 77) where you can identify features, but when you use the 'Tie' tool to open the 'Tie Point Tool' window, the pixels from band 1 (which are stripey garbage) are placed in the window for the LEISA image, which makes it unusable for performing any other activities in qtie. I don't want to make a control point with the pixels in band 1, I want to make control points in bands a, b, and c, and then run a solve (qnet's Tie Point Tool fails in a similar manner).

Rather than dealing with one giant 256-band image, you could pull out individual bands into single-band images, and run qtie on them individually, and then mosaic them together (to get the different bands back into a final, controlled mosaic), or maybe I could try and run qnet/jigsaw on the collection of individual band cubes (not sure if it would vomit on trying to jigsaw single bands from the same image or not).

Running qtie on individual bands and then stacking the mosaicks back together should work (but I haven't done a test recently), but seems like a lot of extra effort, especially if the SPICE data is correctly keeping track of the different spatial geometries in each band.

So, my question is, am I performing the right procedure, and have I found a 'problem' with qtie/qnet (they only provide pixels from band 1 in their 'Tie Point Tool' windows)? Or am I just not 'doing it right?'

[jessemapel]

I wonder if qtie has similar problems with other multi-band instruments

[rbeyer]

I don't do a lot of work with multi-band data like this, so I don't have the background to answer.

[jessemapel]

Sorry, other band dependent instruments.

[rbeyer]

While we ponder the above, this morning I took one band out of the above cube (via cubeatt fr= lsb_0299169338.cub+77 to=lsb_0299169338.77.cub), and brought it in to qtie. I was able to make about 8 control points against the LORRI mosaic, and then performed a Solve. The reported outputs were: Maximum Error = 13.1041, Average Error = 7.96978

This seemed like a lot.

When I took the original image, and the one that had been Solved via qtie and made map projections, this is what I saw:

The one that had been Solved through qtie (on the right above) had features that were significantly warped when compared to the LORRI basemap, while the 'original' (left above) simply appeared translated. This severe warping perhaps explains the large errors reported via the qtie Solve, however, this didn't make sense to me: the bundle-adjusted image should match the basemap better.

At first, I thought I had picked some bad control points or had screwed up with qtie somehow, but then I saw the 'notch.' Remember the 'heart-shaped' Pluto in the raw data because of the thruster firing? If map projection works, you shouldn't see the notch, because the process (if it knows about the thruster firing, etc.) should sort all of that out.

It is clear that the 'original' image had information (the attached SPICE blob, something else?) that allowed cam2map to account for the thruster-firing, and produce a map without a notch, which is why the features on this map seem to just be translated from features on the basemap.

However, it seems like the act of running the Solve function in qtie obliterated the information that accounted for the thruster-firing (or just ignored it), and tried to perform what it thought was a 'simple' bundle adjustment on an image plane that it expected to be geometrically well-behaved, but wasn't (because it didn't account for the thruster-firing), and so it happily did its bast to try and minimize the error, and rewrote the SPICE blob. Then when cam2map comes along, it produces the deformed map.

So even if you could get qtie/qnet to work correctly on a multi-band image generically, there seems to be this issue with the LEISA data, perhaps because of the peculiarity of its thruster-firing, that isn't accounted for either during bundle adjustment or during cam2map.

[jessemapel]

This is an issue we see with high jitter line scan images. The bundle adjust process that qtie uses fits a single polynomial to the entire image and that is not good for images with either high frequency variations (jitter) or dramatic changes (thruster firing). The bundle adjust already has the ability to fix this by only applying a bias polynomial to the original data. This has proven sufficient for adjusting jittery images. It may also be sufficient for adjusting images like this.

So, a possible solution is to add something like jigsaw's overhermite and overexisting parameters to qtie.

[jessemapel]

Another solution could be using piecewise polynomials. This is currently not in the main dev branch and would also require user in out in selecting how many piecewise polynomial segments to use.

[rbeyer]

I figured it might be something like that, and I'm glad to hear that jigsaw has more capability. I was just taking the 'easy way' by hitting the big 'Solve' button in the qtie GUI, since that's where I was. So qtie writes out a control network. Could I just take that control network and run jigsaw with it to get the behavior you're talking about? What parameters would you recommend using with jigsaw for a test like this?

[jessemapel]

I'm not super familiar with the control networks generated by qtie, but you should be able to take this straight into jigsaw. you will want to keep the spacecraft position fixed and solve for just the pointing. I'd suggest CAMSOLVE=ACCELERATIONS OVEREXISTING=YES SPSOLVE=NONE CAMERA_ANGLES_SIGMA=1 CAMERA_ANGULAR_VELOCITY_SIGMA=0.1 CAMERA_ANGULAR_ACCELERATION_SIGMA=0.01 RADIUS=YES POINT_LATITUDE_SIGMA=1000 POINT_LONGITUDE_SIGMA=1000 POINT_RADIUS_SIGMA=1000

Some of these parameters are total guesses. The camera pointing sigmas are probably okay, but you may want to tweak them based on how far off you think the pointing actually is (units are degrees). The point sigmas are also total guesses. I don't know what the ground resolution of this image is, but I'm just guessing 1 km/pixel and used that as the point sigmas. It looks like it's off by more than 1 pixel in most places though, so you may want to increase them. Also, if you think the offset is gross error that is not very complex, you can reduce the pointing solve degree to just 1st by setting CAMSOLVE=VELOCITIES.

[lwellerastro]

@jessemapel types much quicker than I do, but here's my take:

I highly recommend avoiding qtie if at all possible. I have had a multitude of issues with it when trying to work with very squirrelly M3 images including high very high errors. I think the latter had to do with less than optimal data and the fact that I could only solve for camera angles and twist and the data needed more (accelerations/overexisting and spacecraft/overhermite). I also discovered that once the image was updated with qtie, I could not pass it to jigsaw when it was included in a global tie point. There's a post out there on this one.

When processing the THEMIS IR data, I first spiceinit'ed the multi-band image, then extracted band 9 from images for control network generation. Those images were bundled and updated and then had kernels built upon the output. I was then able to successfully apply the kernels via spiceinit to the multi-band cubes and have the updates applied to all the bands. I would do something similar in your situation.

I would also use qnet to build the network, using your mosaic as the Ground Source (be sure to put it in the left window when the edit tool opens so that it is the reference), set the points to either "Fixed" or "Constrained" (be sure to "Set Apriori/Sigmas" on these types of points via qnet), then bundle. I don't know these data at all, but if they are line-scan I would try solving for radius (always seems to help), camera accelerations and twist and setting overexisting=true. If the points are "Fixed", then don't solve for radius - the radius of the point will be pulled from the shape/sphere that was picked up via spiceinit. If using constrained points and setting radius=true, be sure to set point_radius_sigma to something (500 meters is probably a good start...picking out of the air here) and you should also put something down for camera the camera sigmas on how good the pointing is. If the spice is bad you might want to give the bundle room to really adjust the camera - maybe 2 degrees on the angles, 1/2 that for velocities, and half that for accelerations. Once you've bundled you can look at bundleout.txt to see how much the image wanted to correct and tweak the numbers some based on that.

There is no reason to put sigmas on the point latitude/longitude unless you know with some level of certainty that all your point lat/lons should be within some value. Plus this is mute for fixed points and with constrained points you will set those values directly on the point via qnet.

[jessemapel]

Sorry a correction:

The point sigmas should be based on how well you think you picked the ground points. Take how precise you think your were in pixels and then scale that by the pixel resolution in meters. The gross error in the pointing should be accounted for in the camera pointing sigmas.

[lwellerastro]

I don't know about using the control network from qtie directly in qnet. Sounds familar, as if I tried it once and it didn't work for some reason.

@jessemapel, since @rbeyer is tying one image to a ground source his points should be fixed or constrained so using the point_latitude_sigma and point_longitude sigma is not applicable as those are global parameters for FREE points. I have never used those bundling any data set and when I asked Ken when/why you would use them he said that he said almost never. They would be in the case if you knew your data were over the a particular well-know area and wanted to be sure your lat/lon wouldn't stray. But I would just use constrained points for that scenario anyway.

[rbeyer]

@lwellerastro, thank you, your THEMIS process is a very useful analogy.

Okay, my next step will be to ignore qtie and build a control network with qnet, as you have suggested, and see what happens when I jigsaw. I'll find out if the results properly account for the thruster-firing, etc.

From a 'process' perspective, do you think this work-flow is ideal? You have to pull out a band, do some qnet work derive some kernels, and only then can you go back and apply those kernels to the full-stack cube (or cubes). Do you think there is any advantage to being able to select a band from a full-stack cube in qnet itself to work with (rather than manually extracting it)? Maybe the workflow actually is better now, because you are only incurring processing time and storage with a single band image, rather than working with a full-stack cube, which could be more computationally intensive, especially when you're futzing with jigsaw?

[lwellerastro]

From a 'process' perspective, do you think this work-flow is ideal? You have to pull out a band, do some qnet work derive some kernels, and only then can you go back and apply those kernels to the full-stack cube (or cubes).

Yes, this is what I would do, though some of it may be just by habit at this point. Although we did prove with THEMIS that it was absolutely essential to spice, then extract the band in order to get updated kernels that properly applied to the entire stack. Doing it the other way around (extract, then spice the single band) did not work. I think it's because when all bands are present and spice, you are assured of getting the full time range of the image, but (at least in the case of THEMIS), if you extracted first, you would only get the time range of the single band. Not sure this applicable to all instruments though.

I think qtie and qnet have a bug if they are insisting on band one in the editing tool - it would be nice if the user could decide which band to work with in a multi-band file. The entire THEMIS stack would have been fine to work with in the end, but in the case of M3 it was a no brainer to just look for the consistently nicest band and work with that. Plus whenever I update I immediately make a controlled mosaic (before making kernels) to be sure everything looks great, and having a deep cube with data that essentially runs pole to pole would have really tested my patience when running cam2map.

Honestly, I'm not sure I have taken a multi-band cube into jigsaw (maybe Kaguya MI), but I don't think it would care since it's not working with bands and pixels, but rather the spice, point and body information. However, other ISIS overhead could slow things down with a multi-band file.

[rbeyer]

Well, that worked.

What I did:

1. leisa2isis
2. spiceinit
3. cubeatt to pull out one good-contrast band
4. qnet (not qtie!) to create network, set sigmas on points
5. jigsaw (as discussed above)
6. ckwriter to create CK kernel

Then, starting only with a freshly downloaded PDS file and the CK:

1. leisa2isis
2. spiceinit as before, but plus the new CK
3. cam2map

The resulting mosaic has all of the bands, they appear to be correctly registered to one another spatially, and the 'notch' is properly accounted for: geometry managed!

In summary, this turned out to not really be a software issue, and I should have instead posted it on astrodiscuss to ask 'how to do it properly' and I apologize. I am not the first (nor will I be the last) to assume a software problem, when the problem was really between the keyboard and the chair.

The only potentially remaining issue is should qnet/qtie have a band selector? I'm not sure that's even needed because (1) working with a single band as you iterate and experiment with qnet and jigsaw makes the process efficient and (2) as Lynn points out, it is unclear how jigsaw might deal with multiband images.

[jlaura]

@rbeyer Glad that this got worked out for you. We would all benefit from a mini-tutorial on what you did and how you did it. I wonder if you might consider submitting one? I can either get you commit rights on the repo two write to the wiki or you can clone the wiki and submit a PR. That way we can close this issue, but keep the really good usage workflow that was a product of figuring out how to accomplish your task.

[rbeyer]

I might consider it. What kind of 'mini-tutorial' are you looking for, exactly? One on LEISA data, or more generically about control of multi-band images? I don't know that it gets any more 'mini' than the enumerated steps above.

[jessemapel]

Something people have asked for are tutorials on how to work with different data sets. I believe that people are looking for something similar to the workshops that Tammy used to put together, except for a specific data set. A tutorial on how to work with the LEISA data would be helpful.

That said, I think this is a good starting point and making a full up tutorial for this data set will be a significant time investment. A good starting point may be just copying and pasting this to the wiki, or to astrodiscuss. We haven't really figured out how to best store these things.

[lwellerastro]

astrodiscuss has a Tips and Tricks subheading under ISIS - maybe that is place to start putting this sort of thing. However, the workflow @rbeyer outlined above would only be useful for an experienced user. Asking him to fill in the details would be a significant time investment as @jessemapel pointed out, but knowing which programs to use (qnet vs qtie) and how to better deal with a multi-cube image would like help others in similar situations. Better than nothing?

[jlaura]

@lwellerastro Great point. It sounds like a cut/paste contribution into the wiki would be a great start on a document that others could also contribute to at some point in the future if they worked through a similar process.

I say the wiki because we should be seeing all the old ISIS tutorials moving over onto our wiki before the end of the fiscal. They are current in another person's own repo, staged as a test.

[rbeyer]

Seems like once those come over it might be clearer where to put this information and what kind of format you're after. The original issue here really is closed. The secondary issue of moving some of the gained knowledge over into the wiki is a separate thing.