No output or gcp file when running cam_gen

[antonydellavecchia]

When running cam_gen using while working with skysat video files

NASA Ames Stereo Pipeline 2.7.0 Build ID: b844a356b

Built against: NASA Vision Workbench 2.7.0 Build ID: 681b79696 USGS ISIS 4.1.0 Boost C++ Libraries 106800 GDAL 2.0.2 | 20160126 Proj.4 493

i am not getting a gcp-file or an output, but i am also not getting any errors logs, and the only log i get is --> Setting number of processing threads to: 4 .

^ point included, there are the parameters that get passed to the command in case that helps /efs/stereo_procc/Berlin_L1A_AllFrames_20200920/s3_20200920T101514Z/l1a_frames/1284632131.97694254_sc00003_c3_PAN.tif --focal-length 553846.153846 --optical-center 1280 540 --pixel-pitch 1 --height-above-datum 45.0 --gcp-std 1 -o /efs/antony/test.tsai --gcp-file /efs/antony/test.gcp --datum WGS84 --reference-dem /efs/stereo_procc/antony_l1a/generated_dem.tif --frame-index /efs/stereo_procc/Berlin_L1A_AllFrames_20200920/s3_20200920T101514Z/frame_index.csv

this is the line inside the frame_index.csv these are the headers

name, datetime, gsd, sat_az, sat_elev, x_sat_eci_km, y_sat_eci_km, z_sat_eci_km, qw_eci, qx_eci, qy_eci, qz_eci, x_sat_ecef_km, y_sat_ecef_km, z_sat_ecef_km, qw_ecef, qx_ecef, qy_ecef, qz_ecef, bit_dpth, geom, integration_time_ms

and this is the corresponding line to the file

1284632131.97694254_sc00003_c3_PAN,2020-09-20T10:15:13.976943+00:00,1.0516159978009896,64.64277670679789,58.89781195168114, -3997.77716, 691.22320, 5485.65794, 0.22982629, 0.00492334, 0.95239673, 0.20024013, 3891.98217, 1182.94804, 5477.73343, 0.24810289, -0.92754910, -0.21517451, 0.17831866, 16, "POLYGON ((13.4049961811328 52.6462725511156,13.4049862600012 52.6368524715991,13.4457056205799 52.6391812850262,13.445912868603 52.6486402758456,13.4049961811328 52.6462725511156))", 0.9380799531936646

[oleg-alexandrov]

I just ran the tool and it works for me. There is a chance maybe you are using it for some data which it can't handle gracefully, for some reason.

You can try to use it perhaps without --frame-index. Maybe it is parsing something not right from that file. You can specify instead explicitly the four lon-lat corners from your polygon.

This is just to give you an idea of where the problem may be before debugging it in more detail.

Here's one example which I tested and which works for me, together with the output. You can substitute your own image in the call below. Maybe this will help you debug it. If nothing works, we may need to dig deeper:

cam_gen --datum WGS84 --height-above-datum 0 --lon-lat-values '-122.38 37.62 -122.35 37.62 -122.35 37.61 -122.39 37.61' --focal-length 28.968757624607505 --optical-center 18.11581494334796 12.0750534040106 --pixel-pitch 0.0063 image.tif -o run/run.tsai --gcp-file run/run.gcp

--> Setting number of processing threads to: 8

No reference DEM provided. Will use a height of 0 above the datum: Geodetic Datum --> Name: WGS_1984 Spheroid: WGS 84 Semi-major axis: 6378137 Semi-minor axis: 6356752.3142451793 Meridian: Greenwich at 0 Proj4 Str: +proj=longlat +datum=WGS84 +no_defs Writing: run/run.gcp The error between the projection of each ground corner point into the coarse camera and its pixel value: Corner and error: (0 0) 1045.44 Corner and error: (5760 0) 574.421 Corner and error: (5760 3840) 1102.44 Corner and error: (0 3840) 1278.91 Output camera center lon, lat, and height above datum: Vector3(-122.368,37.615,2182.42) Writing: run/run.tsai

[ShashankBice]

Hi @antonydellavecchia, adding on to what Oleg said, I think I get where this is coming from. The L1A all frames sample we got have the bounding box geometry formatting dissimilar to what we got for all the video products (for which this tool was developed). There is a space betweenPOLYGON and the (( : "POLYGON ((13.4049961811328 52.6462725511156,13.4049862600012 52.6368524715991,13.4457056205799 52.6391812850262,13.445912868603 52.6486402758456,13.4049961811328 52.6462725511156))", which is not what we had in videos, and that is what ASP's parser looks for (no space between POLYGON and (( ). Also, we got four coordinates for the four corners for all video samples, but your sample seems to be a closed polygon with five corner values. So a quick workaround should be reformatting the geometry column by skipping the last corner coordinate, and removing the space between POLYGON and (( . Then cam_gen should do its job :)

[antonydellavecchia]

ok thank you I will test this out now, and it does seem to be a problem with the csv, Also, i have an unamed column that is being created I am not sure if this may also be cause an issue.

[Imanflow]

@ShashankBice How is ASP parsing the polygon? Well Know Text (WKT) parsers are capable of handle the space between POLYGON and ((

[oleg-alexandrov]

I parsed that file character-by-character since it is a vendor-specific format so I did not assume it would change.

I looked at your format. Another difference is that, at least how you pasted things, now values which used to be all on one line are on many lines. What I mean is that the format the tool is used to is like this:

1225648264.70823431_sc00004_c1_PAN,2018-11-07T17:50:46Z,1.12461,-148.386,49.1322,-4970.92,-2426.15,4076 .7,-0.686077,0.0485992,-0.723821,-0.0549496,16,"POLYGON((-106.101 39.4783,-106.061 39.4822,-106.066 39. 4674,-106.105 39.4639))"

If you can share with me a copy of your csv file I will add support to it.

[antonydellavecchia]

after creating a string to meet the parser needs i am able to get past the original error and I am now getting an error

from this command /tmp/StereoPipeline-2.7.0-2020-07-29-x86_64-Linux/bin/cam_gen /efs/stereo_procc/berlin_l1a_antony/s3_20200920T101514Z/l1a_frames/1284632135.31540632_sc00003_c1_PAN.tif --focal-length 553846.153846 --optical-center 1280 540 --pixel-pitch 1 --height-above-datum 45.0 --gcp-std 1 -o /efs/stereo_procc/berlin_l1a_antony/cameras_and_gcps/1284632136.21425676_sc00003_c2_PAN_frame_idx.tsai --gcp-file /efs/stereo_procc/berlin_l1a_antony/cameras_and_gcps/1284632136.21425676_sc00003_c2_PAN_frame_idx.gcp --datum WGS84 --reference-dem /efs/stereo_procc/berlin_l1a_antony/generated_dem.tif --frame-index /efs/stereo_procc/berlin_l1a_antony/s3_20200920T101514Z/frame_index.csv --refine-camera --parse-ecef

--> Setting number of processing threads to: 4

Parsed the lon-lat corner values: 13.4039409289347 52.527517255313,13.4036474641219 52.5182291525814,13.4436542963037 52.5200336109424,13.443947604302 52.5293043067974))" Parsed the ECEF camera center in km: 3922.25659 1182.53451 5456.2273399999995. Parsed the ECEF quaternion: 0.24669264 -0.9253648 -0.20796793 0.19897757. Using nodata value: -32768 Parsed camera center (meters): Vector3(3922256.58999999985,1182534.51000000001,5456227.33999999985) Parsed camera quaternion: Q Writing: /efs/stereo_procc/berlin_l1a_antony/cameras_and_gcps/1284632136.21425676_sc00003_c2_PAN_frame_idx.gcp The error between the projection of each ground corner point into the coarse camera and its pixel value:

VW Error: PinholeModel: Projection into pinhole camera is inaccurate.

there might have been an error in how i copied thecsv entry in my last post, this is what the csv shows for this file

name 1284632135.31540632_sc00003_c1_PAN datetime 2020-09-20T10:15:17.315406+00:00 gsd 1.039 sat_az 67.2044 sat_elev 59.9399 x_sat_eci_km -4016.61 y_sat_eci_km 699.926 z_sat_eci_km 5470.8 qw_eci 0.243457 qx_eci 0.00932438 qy_eci 0.949847 qz_eci 0.196044 x_sat_ecef_km 3912.98 y_sat_ecef_km 1182.67 z_sat_ecef_km 5462.84 qw_ecef 0.247146 qx_ecef -0.926097 qy_ecef -0.210184 qz_ecef 0.192576 bit_dpth 16 geom POLYGON ((13.4041309069705 52.5638553872825,13... integration_time_ms 0.64042 geometry POLYGON ((13.4041309069705 52.5638553872825, 1... asp_geometry POLYGON((13.4041309069705 52.5638553872825,13.4038617827606 2.5545113160464,13.4441819097466 52.5565011423936,13.4444031936956 52.5658153807613))

i added the asp_geometry because to remove the fifth point i need to construct the string myself, a shapely polygon will always spit out 5 coordinates to a box polygon. and it does seem to me like the frame_index.csv was created using the geopandas library. I can include it here if you this information isn;t sufficient.

once i got this error though i tried to run the command once again this time without the parse-ecef option and i get this message

-> Setting number of processing threads to: 4 Parsed the lon-lat corner values: 13.4041309069705 52.5638553872825,13.4038617827606 52.5545113160464,13.4441819097466 52.5565011423936,13.4444031936956 52.5658153807613))" Using nodata value: -32768 Writing: /efs/stereo_procc/berlin_l1a_antony/cameras_and_gcps/1284632136.21425676_sc00003_c2_PAN_frame_idx.gcp The error between the projection of each ground corner point into the coarse camera and its pixel value: Corner and error: (0 0) 1373.52 Corner and error: (2560 0) 1364.43 Corner and error: (2560 1080) 1376.73 Corner and error: (0 1080) 1362.9 The error between the projection of each ground corner point into the refined camera and its pixel value: Corner and error: (0 0) 6.31617 Corner and error: (2560 0) 6.3119 Corner and error: (2560 1080) 6.18597 Corner and error: (0 1080) 6.19024 Output camera center lon, lat, and height above datum: Vector3(16.6957,52.6788,-31157.7) Writing: /efs/stereo_procc/berlin_l1a_antony/cameras_and_gcps/1284632136.21425676_sc00003_c2_PAN_frame_idx.tsai

So at this point i am wondering if dropping this command will be alright, i am still working on producing results

[ShashankBice]

This seems fishy to me. I think the order of the polygon is not what we had :) . I say this because if you parse-ecef, then it complains of not being able to project into the pinhole model. When you do not use it, eventhough it runs, it produces a camera which is at -31157.7 m, ie. below the Datum ! Maybe sharing the file with Oleg will help, as he can check it and make the algorithm more robust to meet the new specifications (i.e., to parse correctly the ground footprint, and in the correct order). To also debug on your end, I think @antonydellavecchia, you could use shapely's orient, and then skip the closing polygon coordinate, remove the space (between POLYGON (( ) and then rerun cam_gen. Maybe try with both clockwise and counter-clockwise orientation options. You should get a decent return for one of them. If I seem unclear, maybe this discussion could be of use: https://github.com/icesat2py/icepyx/issues/24

[oleg-alexandrov]

Shashank, thank you, this is very helpful.

About how the .csv file is pasted, I would then like confirmation if all those numbers which show up on separate lines are in fact on the same line in your file. I appreciate you adding clarifications when you pasted things, but for the tool it would be helpful to have the exact file so that later we don't have to do more iterations on this.

[antonydellavecchia]

frame_index.zip

so frame_index.csv is the file we got from planet, and the subset csv is the csv I generated by crop the geo dataframe with a aoi using the geopandas

[antonydellavecchia]

After oriented the polygon in a counter clockwise ordering, starting with bottom corner i still get the projection into pinhole is inaccurate heere's one of the commands and /tmp/StereoPipeline-2.7.0-2020-07-29-x86_64-Linux/bin/cam_gen /efs/stereo_procc/berlin_l1a_antony/s3_20200920T101514Z/l1a_frames/1284632136.79233313_sc00003_c1_PAN.tif --focal-length 553846.153846 --optical-center 1280 540 --pixel-pitch 1 --height-above-datum 45.0 --gcp-std 1 -o /efs/stereo_procc/berlin_l1a_antony/cameras_and_gcps/1284632136.21425676_sc00003_c2_PAN_frame_idx.tsai --gcp-file /efs/stereo_procc/berlin_l1a_antony/cameras_and_gcps/1284632136.21425676_sc00003_c2_PAN_frame_idx.gcp --datum WGS84 --reference-dem /efs/stereo_procc/berlin_l1a_antony/generated_dem.tif --frame-index /efs/stereo_procc/berlin_l1a_antony/s3_20200920T101514Z/frame_index.csv --parse-ecef --refine-camera ted_dem.tif --frame-index /efs/stereo_procc/berlin_l1a_antony/s3_20200920T101514Z/frame_index.csv --parse-ecef --refine-camera --> Setting number of processing threads to: 4 Parsed the lon-lat corner values: 13.4036474641219 52.5182291525814,13.443947604302 52.5182291525814,13.443947604302 52.5293043067974,13.4036474641219 52.5293043067974))" Parsed the ECEF camera center in km: 3922.25659 1182.53451 5456.2273399999995. Parsed the ECEF quaternion: 0.24669264 -0.9253648 -0.20796793 0.19897757. Using nodata value: -32768 Parsed camera center (meters): Vector3(3922256.58999999985,1182534.51000000001,5456227.33999999985) Parsed camera quaternion: Q Writing: /efs/stereo_procc/berlin_l1a_antony/cameras_and_gcps/1284632136.21425676_sc00003_c2_PAN_frame_idx.gcp The error between the projection of each ground corner point into the coarse camera and its pixel value:

VW Error: PinholeModel: Projection into pinhole camera is inaccurate.

[oleg-alexandrov]

I got your frame_index.csv file, thank you. I will take a look tomorrow.

The issue of order of vertices is tricky.

Ideally you can identify this area in Google Earth or Google Maps in Satellite mode. In Google Maps, if you right-click on a location and select "What's here" it will show its lon-and-lat coordinates. You can compare that with Planet's date. Note that cam_gen traverses the corners as 0,0 w,0, w,h, 0,h where w and h are the image width and height.

Note also that you can bypass that .csv file altogether, and specify directly the lon-lat of those corners via --lon-lat-values (see tool's help for specifics). Hopefully after some massaging you will get something which works.

[oleg-alexandrov]

I put a fix for frame_index.csv having 5 lon-lat pairs (first being equal to the last) and it can parse the POLYGON field more robustly. Still no solution for your lon-lat issues not being consistent with what cam_gen wants. I don't understand why it works for some Planet files and not for others.

[antonydellavecchia]

thanks for the string parser fix. are you using some sort of right handle rule to figure out direction of height based on lon lat coordinates? Maybe you can point me to the part of the code where the issue is coming from?

[oleg-alexandrov]

The logic is here: https://github.com/NeoGeographyToolkit/StereoPipeline/blob/master/src/asp/Tools/cam_gen.cc#L121

The idea is to imagine the camera as a frustrum, or like a pyramid with a square base, if you wish. The tip of the pyramid is the camera center. The base is the image. Those four lon-lat corners, together with the height, define a rectangular area on Earth, the base of the pyramid. The tip of the pyramid is up in the air and it is computed based on the base. In other words, the camera is looking down at the Earth, seeing those four corners.

So it is something about the order of those four points. In principle you can debug it with three points only, by invoking cam_gen with --lon-lat-values and --pixel-values. Given three lon-lat pairs, there are 6 ways of ordering them, and you can see which is the right order.

The alternative is, again, to look on Google maps and pick lon-lat values for your image corners based on what you read on the map, and compare with what is in the csv file.

[ShashankBice]

Ok, I experimented a bit, and got plausible results. Here is my basic notebook. Basically I put the ordering of points so that the geographically upper left corner is encountered first, and then others are traversed in clockwise. Might need to be tested with other samples before we have it as a script modification. asp_camgen_video_bugfix.pdf Once I got the updated frame index, I ran this command at a constant height with a dummy image having the same dimensions, as I assume Berlin to be flat, and ok for testing. cam_gen 1284632131.97694254_sc00003_c3_PAN.tif --focal-length 553846.153846 --optical-center 1280 540 --pixel-pitch 1 --height-above-datum 45.0 --gcp-std 1 -o 1284632131.97694254_sc00003_c3_PAN.tsai --gcp-file 1284632131.97694254_sc00003_c3_PAN.gcp --datum WGS84 --frame-index frame_index_asp_format.csv --refine-camera --parse-ecef I got a camera center at plausible position (450 kms, the lowered orbit height):

Corner and error: (0 0) 334398.098899826524
Corner and error: (2560 0) 333595.021325332928
Corner and error: (2560 1080) 333778.591641928768
Corner and error: (0 1080) 334565.159606276487
The error between the projection of each ground corner point into the refined camera and its pixel value:
Corner and error: (0 0) 3.54465628528043863
Corner and error: (2560 0) 3.54935251458755641
Corner and error: (2560 1080) 3.56155811641285913
Corner and error: (0 1080) 3.55677261235549036
Output camera center lon, lat, and height above datum: Vector3(16.8554039297087144,53.509216975990384,459134.04289767408)
Writing: 1284632131.97694254_sc00003_c3_PAN.tsai

Maybe check the updated csv properly on your end @antonydellavecchia ? I am also attaching the updated frame_index file frame_index_asp_format.zip