Digital Globe stereo images not containing XML per image

[daviddemeij]

I have been trying to process Digital Globe stereo images using this tool but we encountered some problems. We have a TIF and RPB file for each image tile (R1C1, R1C2, etc) but no XML file, only one XML file for the entire image.

This resulted in problems when trying to do stereo -t dg (-t rpc does seem to work), I get the following error (which I think is caused by no XML files for the tile being present): terminate called after throwing an instance of 'xercesc_3_1::SAXParseException'

Also when trying to combine the tiles using dg_mosaic according to the example in the documentation: > dg_mosaic 12FEB12053305*TIF --output-prefix 12FEB12053305 --reduce-percent 50 (page 20) does not work because of missing XML files. Error:

Traceback (most recent call last):
  File "/home/ubuntu/StereoPipeline-2.6.1-2018-09-06-x86_64-Linux/libexec/dg_mosaic", line 1081, in <module>
    sys.exit(main())
  File "/home/ubuntu/StereoPipeline-2.6.1-2018-09-06-x86_64-Linux/libexec/dg_mosaic", line 741, in main
    = read_xml( xml_name(args[0]), options, xml_keys)
  File "/home/ubuntu/StereoPipeline-2.6.1-2018-09-06-x86_64-Linux/libexec/dg_mosaic", line 524, in xml_name
    + xml_file_l + ' ' + xml_file_u)
Exception: ('InputError', 'Cannot find any of: ....xml, ...xml, etc')

The XML for the entire image does contain some information for each tile it seems, is there a way in which we could use this information instead?

It contains the following XML tags for each tile:

        <TILE>
    <FILENAME><ULCOLOFFSET><ULROWOFFSET><URCOLOFFSET><URROWOFFSET><LRCOLOFFSET><LRROWOFFSET><LLCOLOFFSET><LLROWOFFSET><ULLON><ULLAT><URLON><URLAT><LRLON<LRLAT><LLLON><LLLAT><ULX><ULY><URX><URY><LRX><LRY><LLX><LLY>       
        </TILE>

[oleg-alexandrov]

Those tools do need the original linescan model in the xml file which you are lacking. So all you can do is use -t rpc and not use dg_mosaic.

I would also check if what you have is raw images rather than orthorectified and georeferenced images, as then the tools won't work either, because orthorectification loses information.

[daviddemeij]

Thank you very much for the quick response, very helpful.

We did check and the images are not orthorectified and we do get results using --t rpc (but they are just not as accurate).

We will request the XML linescan model files from DigitalGlobe.

[daviddemeij]

DigitalGlobe is not aware of any XML line scan model that they can send us and we are also not very clear what information this file includes. Could you give us some more information on what a line scan model is or maybe some pointers on where we can find more about this? Does every stereo image pair usually include these files?

[oleg-alexandrov]

Interesting. As I know, Digital Globe has multiple levels of products, from the most raw to derived. The most raw could be called level 0, maybe. That is the one we need. It should have in the XML file lines like: TLCTIME, EPHEMLIST, ATTLISTList.

In human language, we need their XML files that record the attitude and ephemeris for each image line in their images. And we need the images before they are orthorectified. Later all they lump all that into a single black box model but that is not usable by us.

[daviddemeij]

Thanks again, this is very helpful for our understanding.

The image was taken with the WorldView 3 satellite, is possible that this satellite doesn't collect this line scan information on attitude and ephemeris?

[daviddemeij]

Update: we got some more information from DigitalGlobe. Apparently, this information should be in ephemeris files which are only delivered with processing level 1B (we ordered OR2A) and this processing level is only available when you order the whole image (and not a specific AOI crop as we did).

According to DigitalGlobe "There is no loss of quality or accuracy using the RFM model (using RPC provided with OR2A level) to generate a DSM compared to a rigorous modeling using the ephemeris. It’s rather a point about software interoperability."

[oleg-alexandrov]

Yeah, that makes sense. I guess you can use the RPC model. You can study the IntersectionError file if you call point2dem with the --errorimage flag to see if there are any systematic errors due to cameras. Our bundle_adjust tool can be used to improve the consistency of cameras and then pc_align can align your resulting DEMs to the ground truth. Or you can order the whole image. That won't be cheap, if you have to purchase them.

[dshean]

Yeah, you always want to order L1B stereo if DEM generation is your objective. The OR2A products don't really add much, and you can always generate something equivalent using a simple mapproject command with --datum-offset flag.

Not sure if this is still the case, but in the past ASP had trouble with the OR2A products. This was related to the fact that "Ortho Ready Standard Imagery is projected to a constant base elevation, which is calculated on the average terrain elevation per order polygon or can be supplied by the customer". So they've already warped the two images to the same surface, which is comparable to the transformation in stereo_pprc or the mapproject approach.

But maybe ASP can properly parse and use RPCs from OR2A products at this point. In theory, if we know this average terrain elevation (should be somewhere in metadata), ASP could take this into account and skip initial transform. Alternatively, we could undo their projection, although that would degrade image quality.

In the past, I've advised people in this situation to request that DG deliver the lower-level L1B products instead of the higher-level OR2A products.

[oleg-alexandrov]

Ah, so it is projected images. Likely we can't handle it properly even now. Maybe in the future if we know what the value of the "constant base elevation" is, we could do it. So one indeed may have to get the original data.

[bpurinton]

Hi all,

I know this is an old thread, but I too am having a similar issue. Only the L2A product was delivered on the WV order for a small clipped study area, and these are producing subpar results since I'm unable to use the mapproject tool to project the images onto a preliminary DEM. It's unfortunate that the L1B data are only available for full scenes, since these can be prohibitively expensive compared to clips for a given study area. I note that the product *.XML does contain the map projection information:

<MAP_PROJECTED_PRODUCT>
    <EARLIESTACQTIME>2017-01-24T17:46:04.245314Z</EARLIESTACQTIME>
    <LATESTACQTIME>2017-01-24T17:46:04.245314Z</LATESTACQTIME>
    <DATUMNAME>WE</DATUMNAME>
    <SEMIMAJORAXIS>6.378137000000000e+06</SEMIMAJORAXIS>
    <INVERSEFLATTENING>2.982572235630000e+02</INVERSEFLATTENING>
    <DATUMOFFSETList>
        <DATUMOFFSET>0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00</DATUMOFFSET>
    </DATUMOFFSETList>
    <MAPPROJNAME>UTM</MAPPROJNAME>
    <MAPPROJCODE>1</MAPPROJCODE>
    <MAPZONE>19</MAPZONE>
    <MAPHEMI>S</MAPHEMI>
    <MAPPROJPARAMList>
        <MAPPROJPARAM>0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00</MAPPROJPARAM>
    </MAPPROJPARAMList>
    <PRODUCTUNITS>M</PRODUCTUNITS>
    <ORIGINX>7.691727500957000e+05</ORIGINX>
    <ORIGINY>7.218743750003340e+06</ORIGINY>
    <ORIENTATIONANGLE>0.000000000000000e+00</ORIENTATIONANGLE>
    <COLSPACING>5.000000000000000e-01</COLSPACING>
    <ROWSPACING>5.000000000000000e-01</ROWSPACING>
    <PRODUCTGSD>5.000000000000000e-01</PRODUCTGSD>
    <ULX>7.691727500957000e+05</ULX>
    <ULY>7.218743750003340e+06</ULY>
    <ULH>3.761570000000000e+03</ULH>
    <URX>7.761182501143800e+05</URX>
    <URY>7.218743750003630e+06</URY>
    <URH>3.761570000000000e+03</URH>
    <LRX>7.761182501155300e+05</LRX>
    <LRY>7.208948250003610e+06</LRY>
    <LRH>3.761570000000000e+03</LRH>
    <LLX>7.691727500966700e+05</LLX>
    <LLY>7.208948250003320e+06</LLY>
    <LLH>3.761570000000000e+03</LLH>
    <DEMCORRECTION>Base Elevation</DEMCORRECTION>
    <TERRAINHAE>3.761570000000000e+03</TERRAINHAE>
    <NUMGCP>0</NUMGCP>
</MAP_PROJECTED_PRODUCT>

Including the TERRAINHAE factor, which you suggest may be used to skip the initial transform. I wonder if a hacky solution exists...

Best, Ben

[oleg-alexandrov]

If all you have, it will be impossible to do stereo from this. At the minimum, original camera models would be needed. Then one could run stereo with your two mapprojected, clips, the cameras, and provide ASP with a DEM so it does not complain. The results would then be inaccurate but plausible.

As it is, really, what you have is two ground textures, and we don't know where the spacecraft which made them was. It is not possible to create stereo from this, unfortunately.

I guess the DG folks aren't fools, charging a lot more for stuff which actually can be used.

On Fri, Nov 5, 2021 at 1:37 AM Ben Purinton @.***> wrote:

Hi all,

I know this is an old thread, but I too am having a similar issue. Only the L2A product was delivered on the WV order for a small clipped study area, and these are producing subpar results since I'm unable to use the mapproject tool to project the images onto a preliminary DEM. It's unfortunate that the L1B data are only available for full scenes, since these can be prohibitively expensive compared to clips for a given study area. I note that the product *.XML does contain the map projection information:

2017-01-24T17:46:04.245314Z 2017-01-24T17:46:04.245314Z WE 6.378137000000000e+06 2.982572235630000e+02 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 UTM 1 19 S 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 0.000000000000000e+00 M 7.691727500957000e+05 7.218743750003340e+06 0.000000000000000e+00 5.000000000000000e-01 5.000000000000000e-01 5.000000000000000e-01 7.691727500957000e+05 7.218743750003340e+06 3.761570000000000e+03 7.761182501143800e+05 7.218743750003630e+06 3.761570000000000e+03 7.761182501155300e+05 7.208948250003610e+06 3.761570000000000e+03 7.691727500966700e+05 7.208948250003320e+06 3.761570000000000e+03 Base Elevation 3.761570000000000e+03 0

Including the TERRAINHAE factor, which you suggest may be used to skip the initial transform. I wonder if a hacky solution exists...

Best, Ben

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/NeoGeographyToolkit/StereoPipeline/issues/259#issuecomment-961714069, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKDU3HBX6IIMLA7YWT7TATUKOJUNANCNFSM4HLQETYQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[bpurinton]

Yeah, I figured things won't get much better than this:

Cool that Ames still works on the processed images in any case, but not resulting in the quality we were expecting / hoping for. C'est la vie. My SPOT6 tri-stereo for the area is producing beautiful (albeit lower-res) results. Now just need to write some grants to get full L1B WV strips...

[bwwjohnson]

Hi all, I'm in this situation too. I started out in Agisoft Metashape, then tried ASP and got bad results in both. I have come across this paper which produced DEMs from WV3 OR2A imagery [1] DG say the .STE file gives you the satellite locations for the stereo pair. There is also a .RPB file, a .TIL file, and a .IMD file. I've attached these as text files. Does this change the outlook?

21JUL22060840-P2AS-014162243010_01_P001.IMD.txt 21JUL22060840-P2AS-014162243010_01_P001.RPB.txt 21JUL22060840-P2AS-014162243010_01_P001.STE.txt 21JUL22060840-P2AS-014162243010_01_P001.TIL.txt

[1] https://www.researchgate.net/profile/Fran-Domazetovic/publication/341485924_Assessing_the_Vertical_Accuracy_of_Worldview-3_Stereo-extracted_Digital_Surface_Model_over_Olive_Groves/links/5ed0abc445851529451b759b/Assessing-the-Vertical-Accuracy-of-Worldview-3-Stereo-extracted-Digital-Surface-Model-over-Olive-Groves.pdf])