Closed thareUSGS closed 5 years ago
I can take a look at this. I think the thing to do is to use the swig wrapper / jupyter notebook to confirm that these are loading in a more controlled env. Then we can isolate the issue.
I am pretty sure that the issue is that pfeffernusse has not been updated. We got @Kelvinrr's PR in to get the line scanner updated. Now we just need to get his PR on pfeffernusse in. Then we should be back in sync. Aiming for tonight on this.
GXP still isn't happy with the latest build. There is nothing obvious in the logs failing. The image loads fine, but no sensor model is associated. Have we actually tried the code to use the "load the image" and find the *.json file to assign the associated CSM? I don't recall seeing this tested.
a 16x16 pixel tif and the generic linescan ISD are here: /usgs/shareall/thare/SocetCode/GXP_CSM_build/JSON_CTX_test/
@thareUSGS The plugin currently does that and all of the tests are confirming that it works.
With GXP debug mode on it appears I am getting the new messages. Looks like we might be missing a CSM function called "getImageIdentifier". I hope this isn't just a GXP thing. Here are the errors:
2018/10/30 13:33:53 File SenSocCsm30SensorModel.cpp Line 1361 : INFO : Available CSM: USGS_ASTRO_FRAME_SENSOR_MODEL for image file: D:\GXP_Data\JSON_MDIS_test\EN0221843562M.tif
2018/10/30 13:33:53 File SenSocCsm30SensorModel.cpp Line 1461 : DEBUG : generateCsmSupfile() - ERROR: Unsupported function-UsgsAstroFrameSensorModel::getImageIdentifier
2018/10/30 13:33:53 File SenSocCsm30SensorModel.cpp Line 1461 : DEBUG : populateCsmSupportFile() - ERROR: Unsupported function-UsgsAstroFrameSensorModel::getImageIdentifier
2018/10/30 13:33:53 File v:\deskcf-v4302-build\socetgxp\tap_source\libsrc\sen\sensoc\SenSocCsm30SensorUtilities.cpp Line 397 : DEBUG : initCsmSensorModel( ... ) - Unable to initialize CSM Sensor Model
found a reference, page 131: http://gwg.nga.mil/documents/csmwg/documents/CSM_TRD_Appendix_C_API_Version_3.0__15_November_2010.pdf
in CSM-Cameramodel Frame Model
std::string UsgsAstroFrameSensorModel::getImageIdentifier() const {
throw csm::Error(csm::Error::UNSUPPORTED_FUNCTION,
"Unsupported function",
"UsgsAstroFrameSensorModel::getImageIdentifier");
}
Simply need to implement it, I imagine. It's implemented in line scanner model.
Looks like we can just return an empty string (from the original code). But this implies we need to also set an ID... Still looking.
virtual std::string getImageIdentifier() const;
//> This method returns an identifier to uniquely indicate the imaging
// operation associated with this model.
// This is the primary identifier of the model.
//
// This method may return an empty string if the ID is unknown.
//<
virtual void setImageIdentifier(
const std::string& imageId,
csm::WarningList* warnings = NULL);
//> This method sets an identifier to uniquely indicate the imaging
// operation associated with this model. Typically used for models
// whose initialization does not produce an adequate identifier.
//
// If a non-NULL warnings argument is received, it will be populated
// as applicable.
//<
From original code. Looks we use to just return the image's filename:
// Translate the ISD to state data
if( astroLsIsd.m_image_id.size() > 1 && astroLsIsd.m_image_id != "UNKNOWN")
{
state.m_ImageIdentifier = astroLsIsd.m_image_id;
}
else
{
state.m_ImageIdentifier = img_rel_name;
}
where image_rel_name was previously
std::string tmp_file_name = image_support_data.filename();
size_t locDelim = tmp_file_name.find_last_of(DIR_DELIMITER_STR);
size_t length = tmp_file_name.size();
img_rel_name = tmp_file_name.substr(locDelim + 1, length);
and
#ifdef _WIN32
# define DIR_DELIMITER_STR "\\"
#else
# define DIR_DELIMITER_STR "/"
#endif
After ID as filename implemented, framer reporting two more non-implemented functions (getReferencePoint, getParameterCovariance).
2018/10/31 11:51:23 File SenSocCsm30SensorModel.cpp Line 1967 : ERROR : ERROR: UsgsAstroFrameSensorModel::getReferencePoint - Unsupported function
2018/10/31 11:51:23 File SenSocCsm30SensorModel.cpp Line 2025 : ERROR : ERROR: UsgsAstroFrameSensorModel::getParameterCovariance - Unsupported function
2018/10/31 11:51:23 File v:\deskcf-v4302-build\socetgxp\tap_source\libsrc\sen\sensoc\SenSocCsm30SensorUtilities.cpp Line 397 : DEBUG : initCsmSensorModel( ... ) - Unable to initialize CSM Sensor Model
closer... non-implemented functions (getPedigree , getSensorIdentifier)
2018/10/31 12:23:44 File SenSocCsm30SensorModel.cpp Line 1679 : ERROR : ERROR: UsgsAstroFrameSensorModel::getPedigree - Unsupported function
2018/10/31 12:23:44 File SenSocCsm30SensorModel.cpp Line 1856 : ERROR : ERROR: UsgsAstroFrameSensorModel::getSensorIdentifier - Unsupported function
2018/10/31 12:23:44 File v:\deskcf-v4302-build\socetgxp\tap_source\libsrc\sen\sensoc\SenSensorModelSocetSet.cpp Line 967 : ERROR : init( ... ) - unable to create sensor model for C:\Users\thare-pr\AppData\Local\Temp\3\EN0221843562M_642761867.sup
looks like all functions are available, but can't init. Not a very helpful error.
2018/10/31 13:15:34 File SenSocCsm30SensorModel.cpp Line 1361 : INFO : Available CSM: USGS_ASTRO_FRAME_SENSOR_MODEL for image file: D:\GXP_Data\JSON_MDIS_test\EN0221843562M.tif
2018/10/31 13:15:34 File v:\deskcf-v4302-build\socetgxp\tap_source\libsrc\sen\sensoc\SenSensorModelSocetSet.cpp Line 967 : ERROR : init( ... ) - unable to create sensor model for C:\Users\thare-pr\AppData\Local\Temp\3\EN0221843562M_645867635.sup
@thareUSGS Does C:\Users\thare-pr\AppData\Local\Temp\3\EN0221843562M_645867635.sup
exist? Can you dump it's contents here?
Not helpful just yet. Again just shows "can't init". Search for "EN0221843562M"
@thareUSGS Does
C:\Users\thare-pr\AppData\Local\Temp\3\EN0221843562M_645867635.sup
exist? Can you dump it's contents here?
Unfortunately, it was never created. We tested a CTX with similar "can't init" results. I am going to compare some of the original LS code with the updated code to see if something sticks out.
So, I am looking at the logs:
Line 1662 : INFO : d:\gxp_data\json_mdis_test\en0221843562m.tif is mapped to \\IGSWZAWGWSAG012\ddrive\gxp_data\json_mdis_test\en0221843562m.tif
Got it, the image that is being loaded is a tif (or is Socet pulling the .IMG and adding .tif)?
INFO : 651277336 is image id for D:/GXP_Data/JSON_MDIS_test/EN0221843562M.tif
Is Socet assigning this? Our imageId would be: EN0221843562M
2018/10/31 14:45:56 File v:\deskcf-v4302-build\socetgxp\tap_source\libsrc\lsimpexp\LsAds40Helper.cpp Line 206 : ERROR : Unable to find sup file corresponding to D:\GXP_Data\JSON_MDIS_test\EN0221843562M.tif
This line does not look our code is trying to load the side car, but that LsAds40Helper.cpp
is trying to load a sup file. Additionally, look at the mapping to drive location - someone is forcing capitalization in the log (not us).
Since we can load our sensor/sidecar without error via the bindings, where we know exactly what the order of operations is, I wonder if socet is actually operating the way we think it is?
For this thread, speaking to our Framer looks like we still have a couple missing functions.
2019/02/06 13:58:08 File SenSocCsm30SensorModel.cpp Line 1461 : DEBUG : generateCsmSupfile() - ERROR: Unsupported function-UsgsAstroFrameSensorModel::getImageIdentifier
2019/02/06 13:58:08 File SenSocCsm30SensorModel.cpp Line 1461 : DEBUG : populateCsmSupportFile() - ERROR: Unsupported function-UsgsAstroFrameSensorModel::getImageIdentifier
2019/02/06 13:58:08 File SenFSensorModelFactory.cpp Line 1623 : DEBUG : createSensorModel( ... ) - Unable to create CSM30 Sensor Model
Speaking to the linescanner (should split these tickets), GXP is recognizing the ISD json file (specifically for an ISIS3 cube called J03_045994_1986_XN_18N282W.json), but it states not converging thus the camera isn't instantiated. The line-scanner was just patched with #162
J03_045994_1986_XN_18N282W.json.txt
2019/02/06 13:44:42 File SenSocCsm30SensorModel.cpp Line 449 : ERROR : ERROR: UsgsAstroLsSensorModel::groundToImage - Did not converge.
2019/02/06 13:44:42 File SenSocCsm30SensorModel.cpp Line 449 : ERROR : ERROR: UsgsAstroLsSensorModel::groundToImage - Did not converge.
2019/02/06 13:44:42 File SenSocCsm30SensorModel.cpp Line 449 : ERROR : ERROR: UsgsAstroLsSensorModel::groundToImage - Did not converge.
2019/02/06 13:44:42 File SenSocCsm30SensorModel.cpp Line 449 : ERROR : ERROR: UsgsAstroLsSensorModel::groundToImage - Did not converge.
Note I had to rename the json file to a *.txt to get it to upload.
@thareUSGS We can add some more information to the error message to hopefully debug this.
I'll take the framer and have a look tonight or tomorrow.
I just checked out the latest pull from the repo and I am not seeing an issue with the state on the Ls. Specifically: https://gist.github.com/jlaura/8ccd680a10f9afd1bde35e5015cc6139
So the GXP failure is likely not due to a reversion in the state code, unfortunately.
MDIS images are now instantiating correctly, but from there GXP is doing some weird stuff with them. It is calling imageToGround
with a height of about -3,000,000 m which is greater than the radius of the body!
I checked with a height of 0 in a Jupyter notebook and it looks okay.
https://gist.github.com/jessemapel/baa03c0804f71cc284d8c7ed0bd8695a
Attached is a zip folder with the cubes, our logs, GXP's logs, and some screen shots of Socet GXP. There's also similar for a DAWN FC image.
Note that the JSON ISDs in the zip directories have logging paths setup for my shareall directory. If you would like to test them yourself, change the log_file
keyword to where you want you log files. Note that any Windows paths need to use double \
characters.
Hi. Megan here from LROC. We contacted GXP and found out that there API is hard-coded to WGS84, probably explaining your issues.
@meganhenriksen That could explain the problems we're seeing. We did get CTX images to work successfully in Socet GXP, though our linescan code is based on code from BAE.
@thareUSGS has mentioned that with 3.0.3 CSM supports a generic bi-axial ellipsoid (with the Elliopsoid class), but we haven't called anything related to that.
Just noticed, the LS inherits from RasterGM
and SettableEllipsoid
, but the framer only inherits from RasterGM
!
@meganhenriksen Thank you! Been trying to figure this out for the last few weeks.
Good find. I wasn't always sure if we needed our own CSM framer in GXP, but because of their apparent lock into WGS84 🌍 , this should fix framers for us planetary folks 🌝.
This is fixed as of #228
I have two testing images for MDIS NAC and CTX with ISDs from pfeffernusse.
/usgs/shareall/thare/SocetCode/GXP_CSM_build/CTX_test/ and /usgs/shareall/thare/SocetCode/GXP_CSM_build/MDIS_test/
I converted to Tiff and ISIS cubes for GXP testing.
In the log, during "debug" startup, I get the CSM success for the latest usgscam.dll Succeeded to load CSM plugin library: C:\Program Files\BAE SYSTEMS\SOCET GXP 4.3.0\bin\smplugins\csmplugins\3.0.3\usgscsm.dll
Image are loaded fine and the auto pyramids and histogram files are run, but the images are set to no camera model.
I don't see anything specific to our code in the logs besides a generic: File SenFSensorModelFactory.cpp Line 646 : DEBUG : createSensorModel( ... ) - Unable to create CSM or CSM30 Sensor Model
which implies it ran through all the CSM dlls and failed on all of them. GXP doesn't report failure per CSM. I do know if a CSM is found, it will report that as a success in the log. For example, with the original CSM, it will report:
_SenSocCsm30SensorModel.cpp Line 1361 : INFO : Available CSM: USGS_ASTRO_LINE_SCANNER_SENSORMODEL for image file: ...
I wonder if someone can try latest code on these images and available ISD json files.