`cam_gen` VW Error: OpticalBarModel::point_to_pixel: Unable to project point into camera.

[iamtekson]

I am trying to process the KH-9 satellite imagery by following the official tutorial here. I successfully mosaic my images and now I am trying to generate the camera file. I wrote my sample_kh9_for.tsai file as below,

VERSION_4
OPTICAL_BAR
image_size = 68460 32409
image_center = 3430 16204.5
pitch = 7.0e-06
f = 1.5
scan_time = 0.7
forward_tilt = 0.174533
iC = -1053926.8825477704 5528294.6575468015 3343882.1925249361
iR = -0.96592328992496967 -0.16255393156297787 0.20141603042941184 -0.23867502833024612 0.25834753840712932 -0.93610404349651921 0.10013205696518604 -0.95227767417513032 -0.28834146846321851
speed = 8000
mean_earth_radius = 6371000
mean_surface_elevation = 928
motion_compensation_factor = 1
scan_dir = right

I am using SRTM dem as a reference dem (WGS84 coordinate system). I run the command below to generate the cam_gen file,

cam_gen --sample-file sample_for.tsai --camera-type opticalbar --lon-lat-values '40.817 44.938 42.329 44.752 41.804 42.608 40.339 42.784' for.tif --reference-dem srtm.tif --refine-camera -o for.tsai

When I run the above function it outputs the below information and terminate with an error,

--> Setting number of processing threads to: 4
Using nodata value: -3.4028234663852886e+38
The error between the projection of each ground corner point into the coarse camera and its pixel value:
Corner and error: (0 0) 31356.204888916091
Corner and error: (68574 0) 29532.270234978227
Corner and error: (68574 32544) 29629.527235721198
Corner and error: (0 32544) 27782.348147623663

VW Error: OpticalBarModel::point_to_pixel: Unable to project point into camera.

But when I change the focal length to 0.61000001430511475 (This is the focal length for KH4B imagery), it runs successfully and creates a for.tsai file. Is this the problem with focal length or I am missing something? Can anyone please help me?

[oleg-alexandrov]

Given that you used:

image_size = 68460 32409

Then why

image_center = 3430 16204.5

The first number should be 10x bigger, I think, as it should be roughly the image midpoint.

Overall this functionality is rather fragile, but I hope it will work for you if you adjust the center.

On Fri, May 12, 2023 at 6:38 AM Tek Kshetri @.***> wrote:

I am trying to process the KH-9 satellite imagery by following the official tutorial here https://stereopipeline.readthedocs.io/en/latest/examples/historical.html#kh9. I successfully mosaic my images and now I am trying to generate the camera file. I wrote my sample_kh9_for.tsai file as below,

VERSION_4 OPTICAL_BAR image_size = 68460 32409 image_center = 3430 16204.5 pitch = 7.0e-06 f = 1.5 scan_time = 0.7 forward_tilt = 0.174533 iC = -1053926.8825477704 5528294.6575468015 3343882.1925249361 iR = -0.96592328992496967 -0.16255393156297787 0.20141603042941184 -0.23867502833024612 0.25834753840712932 -0.93610404349651921 0.10013205696518604 -0.95227767417513032 -0.28834146846321851 speed = 8000 mean_earth_radius = 6371000 mean_surface_elevation = 928 motion_compensation_factor = 1 scan_dir = right

I am using SRTM dem as a reference dem (WGS84 coordinate system). I run the command below to generate the cam_gen file,

cam_gen --sample-file sample_for.tsai --camera-type opticalbar --lon-lat-values '40.817 44.938 42.329 44.752 41.804 42.608 40.339 42.784' for.tif --reference-dem srtm.tif --refine-camera -o for.tsai

When I run the above function it outputs the below information and terminate with an error,

--> Setting number of processing threads to: 4 Using nodata value: -3.4028234663852886e+38 The error between the projection of each ground corner point into the coarse camera and its pixel value: Corner and error: (0 0) 31356.204888916091 Corner and error: (68574 0) 29532.270234978227 Corner and error: (68574 32544) 29629.527235721198 Corner and error: (0 32544) 27782.348147623663

VW Error: OpticalBarModel::point_to_pixel: Unable to project point into camera.

But when I change the focal length to 0.61000001430511475 (This is the focal length for KH4B imagery), it runs successfully and creates a for.tsai file. Is this the problem with focal length or I am missing something? Can anyone please help me?

— Reply to this email directly, view it on GitHub https://github.com/NeoGeographyToolkit/StereoPipeline/issues/399, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKDU3CR7HDNEW4QWDOE2PTXFY4NTANCNFSM6AAAAAAX7RQQCI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[iamtekson]

Thank you very much for the quick response. I just corrected the image_center as 34230 16204.5 but still the same issue. I am wondering why it is working if I change f=0.61000001430511475. Is it okay to change or the focal length for the KH9 is standard as 1.5?

Also another thing is, while processing the second pair of image (aft.tif), it is compulsory to change scan_dir = left, forward_tilt = - 0.174533 and motion_compensation_factor = -1?

[oleg-alexandrov]

Thank you very much for the quick response. I just corrected the image_center as 34230 16204.5 but still the same issue. I am wondering why it is working if I change f=0.61000001430511475. Is it okay to change or the focal length for the KH9 is standard as 1.5?

I am not sure. I did not write this particular software. I know that we did a reasonable job at implementing the optical bar model, but I don't think we figured out reliably how to find the correct parameters for it. What may have worked for the tutorial example may not work for a different dataset.

I added a disclaimer about that to https://stereopipeline.readthedocs.io/en/latest/examples/historical.html#declassified-satellite-images-kh-9 (it may take a while for the doc to be updated).

Also another thing is, while processing the second pair of image (aft.tif), it is compulsory to change scan_dir = left, forward_tilt = - 0.174533 and motion_compensation_factor = -1?

I don't know either. You may want to read the paper describing the optical bar model, and see the documentation for your particular testcase.

In short, sorry. The support for this sensor is experimental. We played around a bit, figured out the model, but did not have enough time to go from a prototype to a functioning system.

Likely things can work if you play enough with numbers, but it may take time, and there's no guarantee on final accuracy.

[iamtekson]

Thank you very much for the reply. I will try to play with these parameters and see how it works. But it would be really great if someone could maintain and make the KH imagery pipeline functional.

Appreciate your time and contribution. Thank you!

[oleg-alexandrov]

Yes, it would be nice if we could maintain it. Our funding usually dictates what we work on, and have had no funding for historical image processing.

[ScottMcMichael]

I am wondering why it is working if I change f=0.61000001430511475. Is it okay to change or the focal length for the KH9 is standard as 1.5?

A source I referenced while setting up the camera configs specified a 60 inch focal length which is equivalent to 1.5 meters. If half that length is working for you it could mean that the source I used was incorrect, that there were multiple camera configurations used, or that the altered focal length is compensating for an error somewhere else in our camera model.

Also another thing is, while processing the second pair of image (aft.tif), it is compulsory to change scan_dir = left, forward_tilt = - 0.174533 and motion_compensation_factor = -1?

These parameters are supposed to represent that the two cameras rotate in opposite directions and are tilted at opposite angles.

As Oleg said, using ASP for this camera is unfortunately a bit of a research project. Using the limited documentation available I was able to get decent results with our model but never good enough to confirm that all aspects of the camera were being modeled properly.

[iamtekson]

Yes, it would be nice if we could maintain it. Our funding usually dictates what we work on, and have had no funding for historical image processing.

Yes, I would be very happy to maintain it but I am completely beginner with this package. I need to invest much effort and time exploring this package. Anyway, I will explore the package and contribute if I can.

A source I referenced while setting up the camera configs specified a 60 inch focal length which is equivalent to 1.5 meters. If half that length is working for you it could mean that the source I used was incorrect, that there were multiple camera configurations used, or that the altered focal length is compensating for an error somewhere else in our camera model.

But when I put 1.5, the cam_gen was unable to generate the camera_file. Later I used f=0.304 as I found in this paper Table 1, I got the strange result (attached below). I feel like this result is due to improper setting of camera file.

These parameters are supposed to represent that the two cameras rotate in opposite directions and are tilted at opposite angles.

It was suggested in documentation but not clearly.

@oleg-alexandrov, @ScottMcMichael do you suggest any other package or any methodology for processing these historical imagery?

[dshean]

@adehecq's paper is specific to the Hexagon mapping cameras, though some of the underlying workflows with ASP are similar. The mapping cameras (9x18” frame camera) were very different than the panoramic cameras specified in your camera model file, and in many ways, are easier to process. I believe you are trying to process the KH4B panoramic images? Can you provide more information on the input images you’re using?

@ShashankBice and I finally got around to a public release of our early experiments with ASP for the panoramic (optical bar) camera systems: https://github.com/uw-cryo/declass_stereo. Likely some useful information there.

This remains an active research interest, and we would love to have more community support to improve these tools, as we haven’t had the time/funding to focus on them in recent years. Even some basic updates to the existing scripts and doc for the latest ASP versions would be valuable, and we welcome input from others considering using ASP to process images this invaluable archive.

[ScottMcMichael]

I am wondering why it is working if I change f=0.61000001430511475. Is it okay to change or the focal length for the KH9 is standard as 1.5?

A source I referenced while setting up the camera configs specified a 60 inch focal length which is equivalent to 1.5 meters. If half that length is working for you it could mean that the source I used was incorrect, that there were multiple camera configurations used, or that the altered focal length is compensating for an error somewhere else in our camera model.

I realize now a third possibility is that the source confused inches and centimeters =p

[iamtekson]

I believe you are trying to process the KH4B panoramic images? Can you provide more information on the input images you’re using?

I am trying for KH9 imagery (declassified level 2) dataset downloaded from USGS earth explorer. The metadata of both imagery as below,

Entity ID   DZB1213-500238L001001
Acquisition Date    1977/10/07
Mission 1213-5
Frame   1
Image Type  1
Camera Type KH-9 Lower Resolution Mapping
Camera Resolution   20 to 30 feet
Film Type   9 inch by 18 inches
Generation  3
Segment Count   2
Polarity    Negative
Operations Number   00238
Down Load Available 1
Center Latitude 43°46'22.80"N
Center Longitude    41°19'08.40"E
NW Corner Lat   44°56'16.80"N
NW Corner Long  40°49'01.20"E
NE Corner Lat   44°45'07.20"N
NE Corner Long  42°19'44.40"E
SE Corner Lat   42°36'28.81"N
SE Corner Long  41°48'14.40"E
SW Corner Lat   42°47'02.40"N
SW Corner Long  40°20'20.40"E
Center Latitude dec 43.773
Center Longitude dec    41.319
NW Corner Lat dec   44.938
NW Corner Long dec  40.817
NE Corner Lat dec   44.752
NE Corner Long dec  42.329
SE Corner Lat dec   42.608
SE Corner Long dec  41.804
SW Corner Lat dec   42.784
SW Corner Long dec  40.339

Entity ID   DZB1213-500238L002001
Acquisition Date    1977/10/07
Mission 1213-5
Frame   2
Image Type  1
Camera Type KH-9 Lower Resolution Mapping
Camera Resolution   20 to 30 feet
Film Type   9 inch by 18 inches
Generation  3
Segment Count   2
Polarity    Negative
Operations Number   00238
Down Load Available 1
Center Latitude 43°07'51.60"N
Center Longitude    41°07'55.20"E
NW Corner Lat   44°17'41.99"N
NW Corner Long  40°37'51.60"E
NE Corner Lat   44°06'43.20"N
NE Corner Long  42°07'37.20"E
SE Corner Lat   41°58'01.20"N
SE Corner Long  41°37'01.20"E
SW Corner Lat   42°08'24"N
SW Corner Long  40°09'57.60"E
Center Latitude dec 43.131
Center Longitude dec    41.132
NW Corner Lat dec   44.295
NW Corner Long dec  40.631
NE Corner Lat dec   44.112
NE Corner Long dec  42.127
SE Corner Lat dec   41.967
SE Corner Long dec  41.617
SW Corner Lat dec   42.14
SW Corner Long dec  40.166

Looks like the new package you mentioned https://github.com/uw-cryo/declass_stereo is the useful one. I will have a look.

[dshean]

OK. Yes, those are the Hexagon (KH-9) mapping cameras. You need to follow the workflow in @adehecq's paper, and unfortunately, that code is not yet public. The ASP example for the optical bar camera you're attempting to use won't work with these mapping (frame) camera images. For the latter, you need to define a tsai (or other frame camera) model with more traditional interior geometry and optical distortion coefficients.

[dshean]

If you can identify suitable panoramic or Optical Bar camera images from the same time period, then you should be able to follow the ASP example in the doc our the declass_stereo repo.

[iamtekson]

@dshean Thank you very much for the information. I thought the example code is from his paper.

I think it is better to communicate with @adehecq and would be really helpful for me if he can share his code.

[iamtekson]

Does this package provides the utility to process KH9 dataset?

[adehecq]

Hi Tek, Just came across this discussion thanks to @dshean. Sorry I don't check that channel so often anymore since I have to focus on other priorities. As @dshean mentionned, the data you're trying to process is the KH-9 mapping camera (MC) while the tutorial you looked at is for the panoramic camera. I checked the doc, and it's stated at the beginning: "The frame camera is a normal pinhole model so this example describes how to set up the panoramic cameras for processing.". However, maybe this should be made clearer @oleg-alexandrov ? You could use underline/bold and briefly state that the mapping camera belongs to the "Declass 2" dataset (ID starts with DZB12) on Earth Explorer while the panoramic camera belongs to the "Declass 3" data set and additional criteria "Camera Resolution: 2 to 4 feet" (I believe all IDs start with D3C12).

The workflow that I developed for the KH-9 MC data is not described in ASP's documentation, essentially because it's a lot more complex than just a few commands to run. To give you an idea, I run ipfind/ipmatch with customized options, I run several iterations of bundle_adjust to get a reasonable camera and I run several iterations of parallel_stereo with coregistration using pc_align to refine my camera intrinsics and or extrinsics parameters. I will answer to your email to provide you some more details. My workflow is designed to work even with very nasty images, but for most cases, a simple workflow could work and it would be nice to include this in ASP's documentation. Maybe with your input @iamtekson, we could come up with a simple workflow.

[oleg-alexandrov]

Yeah, we glossed over the fact that KH-9 has both frame and optical bar sensors, and they need different approaches. I expanded https://stereopipeline.readthedocs.io/en/latest/examples/historical.html#declassified-satellite-images-kh-9 (hard reload needed).

[iamtekson]

Hi @adehecq, Thank you very much for the reply here. Sure, I will be very happy to contribute on the ASP's documentation about processing the KH9 imagery.

Since I am beginner with the stereo imagery processing and ASP, I haven't noticed that I followed the wrong camera model and wrong guideline. But definitely keen to invest my time on it make a proper documentation.

[iamtekson]

Thank you everyone for your suggestion and inputs. I have processed my stereo-pair using Heximap package. I will revisit to this issue if there is any update.

[oleg-alexandrov]

Glad you found something that works for you.