search

DOI-USGS / ISIS3

Integrated Software for Imagers and Spectrometers v3. ISIS3 is a digital image processing software package to manipulate imagery collected by current and past NASA and International planetary missions.
https://isis.astrogeology.usgs.gov
Other
199 stars 168 forks source link

Serve ORex Kernels #4060

Closed jlaura closed 3 years ago

jlaura commented 4 years ago

Description
Both #4054 and #4058 show a community desire to be able to use ISIS to work with ORex data. Currently, these kernels are not available within the ISIS ecosystem. This enhancement request asks that the kernels be made available.

I see that ORex kernels are available via NAIF here. I do not know if these are the kernels we would want in a format that is immediately usable (so I do not have a handle on the scope of effort required to make this happen).

Example
Spiceinit works without throwing an error.

jessemapel commented 3 years ago

OCams data can be found at the small bodies node: https://sbn.psi.edu/pds/resource/orex/ocams.html we will likely need to check that we support the latest release of data in the ingestion and sensor model.

jessemapel commented 3 years ago

After more discussion, the latest kernels need to be checked with the latest image data. There is likely some amount of issues with the IAK and the camera model that will need to be addressed.

For whomever works on this, I have some makedb scripts and kerneldbs from the mission team that should be useful.

jessemapel commented 3 years ago

osirisrex.tar.gz

The attached tarball has the osirisrex data area configuration from @KrisBecker in it. You'll want to use the kernel.db and makedb files located inside the kernels directories alongside the downloaded kernels. The scripts in there are for more complex kernel management that you can take a look at too.

Kelvinrr commented 3 years ago

@jessemapel Kernels are in using @KrisBecker's makedb scripts and kernel.db files. Where would be the best place to get the new images? I'm sure I asked this before and forgot to write it down...

jessemapel commented 3 years ago

@Kelvinrr You can get data from the Small Bodies Node: https://sbn.psi.edu/pds/resource/orex/ocams.html

Kelvinrr commented 3 years ago

@jessemapel Thats what I figured, thanks.

jessemapel commented 3 years ago

I would recommend data from orbit b or recon b

KrisBecker commented 3 years ago

If you want to let me know which images you select, I will run phocube and caminfo in our system to provide a relative comparison.

Kelvinrr commented 3 years ago

@KrisBecker Full caminfo from 20190608T005825S208_map_iofL2pan.cube derived from https://sbnarchive.psi.edu/pds4/orex/orex.ocams/data_calibrated/orbit_b/20190608T005825S208_map_iofL2pan.fits

Object = Caminfo
  Object = Parameters
    Program     = caminfo
    IsisVersion = "4.3.0  | 2020-12-21"
    RunDate     = 2020-12-23T21:32:50
    IsisId      = OsirisRex/MapCam/3/0613227461.18586
    From        = 20190608T005825S208_map_iofL2pan.cub
    Lines       = 1024
    Samples     = 1024
    Bands       = 1
  End_Object

  Object = Geometry
    BandsUsed                  = 1
    ReferenceBand              = 1
    OriginalBand               = 1
    Target                     = Bennu
    StartTime                  = 2019-06-08T00:58:25.1174066
    EndTime                    = 2019-06-08T00:58:25.1174066
    CenterLine                 = 512.0
    CenterSample               = 512.0
    CenterLatitude             = 39.356959402519
    CenterLongitude            = 120.48019375114
    CenterRadius               = 263.47693503477
    RightAscension             = 149.06938037891
    Declination                = 73.980788554424
    UpperLeftLongitude         = 184.8165775832
    UpperLeftLatitude          = 3.724184552316
    LowerLeftLongitude         = NULL
    LowerLeftLatitude          = NULL
    LowerRightLongitude        = NULL
    LowerRightLatitude         = NULL
    UpperRightLongitude        = 333.58561956185
    UpperRightLatitude         = 61.073494835369
    PhaseAngle                 = 93.753742553614
    EmissionAngle              = 43.504634305712
    IncidenceAngle             = 51.137108669359
    NorthAzimuth               = 328.85727466186
    OffNadir                   = -0.069294598991746
    SolarLongitude             = 115.53347331512
    LocalTime                  = 14.299902614028
    TargetCenterDistance       = 6.441179989244
    SlantDistance              = 6.2529327012405
    SampleResolution           = 0.42452019137815
    LineResolution             = 0.42452019137815
    PixelResolution            = 0.42452019137815
    MeanGroundResolution       = 0.42914142234113
    SubSolarAzimuth            = 92.684951179335
    SubSolarGroundAzimuth      = 228.98001281241
    SubSolarLatitude           = 2.2167791149862
    SubSolarLongitude          = 85.981654540718
    SubSpacecraftAzimuth       = 266.57389247857
    SubSpacecraftGroundAzimuth = 53.290083778222
    SubSpacecraftLatitude      = 51.079668274721
    SubSpacecraftLongitude     = 182.07154617745
    ParallaxX                  = -0.56734865903198
    ParallaxY                  = 0.76088175727065
    ShadowX                    = 0.8144679215095
    ShadowY                    = -0.93627831900772
    HasLongitudeBoundary       = TRUE
    HasNorthPole               = TRUE
    HasSouthPole               = FALSE
    ObliqueSampleResolution    = 0.58528781369234
    ObliqueLineResolution      = 0.58528781369234
    ObliquePixelResolution     = 0.58528781369234
    ObliqueDetectorResolution  = 0.58528781369234
  End_Object
End_Object
End
Kelvinrr commented 3 years ago

I imagine we can close this out with a PR setting the default IsisPreferences to point to the new data area? Is there a justification for keeping the old data_local path?

KrisBecker commented 3 years ago

I would perhaps keep the local directory in some archive. It might be useful for historical regression testing or comparisons.

But you need to update IsisPreferences to the correct path in the ISIS distribution. We have OsirisRex = $ISIS3DATA/osirisrex.

KrisBecker commented 3 years ago

@Kelvinrr Below are the basic commands that I believe you ran on the OCAMS MapCam image take during Orbit B phase:

wget -P . https://sbnarchive.psi.edu/pds4/orex/orex.ocams/data_calibrated/orbit_b/20190608T005825S208_map_iofL2pan.fits
ocams2isis from=20190608T005825S208_map_iofL2pan.fits to=20190608T005825S208_map_iofL2pan.cub
spiceinit from=20190608T005825S208_map_iofL2pan.cub
caminfo from=20190608T005825S208_map_iofL2pan.cub to=20190608T005825S208_map_iofL2pan.ellipsoid.pvl

Here is the caminfo output. Note UA/ISIS version is not adding the oblique emission/detector angles:

Object = Caminfo
  Object = Parameters
    Program     = caminfo
    IsisVersion = "3.6.1  | 2019-03-14"
    RunDate     = 2020-12-24T00:31:05
    IsisId      = OsirisRex/MapCam/3/0613227461.18586
    From        = 20190608T005825S208_map_iofL2pan.cub
    Lines       = 1024
    Samples     = 1024
    Bands       = 1
  End_Object

  Object = Geometry
    BandsUsed                  = 1
    ReferenceBand              = 1
    OriginalBand               = 1
    Target                     = Bennu
    StartTime                  = 2019-06-08T00:58:25.1174066
    EndTime                    = 2019-06-08T00:58:25.1174066
    CenterLine                 = 512.0
    CenterSample               = 512.0
    CenterLatitude             = 39.364046351634
    CenterLongitude            = 120.58567534094
    CenterRadius               = 263.48372446101
    RightAscension             = 149.07911245571
    Declination                = 73.981202394152
    UpperLeftLongitude         = 184.89884392384
    UpperLeftLatitude          = 3.574074548412
    LowerLeftLongitude         = NULL
    LowerLeftLatitude          = NULL
    LowerRightLongitude        = NULL
    LowerRightLatitude         = NULL
    UpperRightLongitude        = 333.42715769106
    UpperRightLatitude         = 60.9779417995
    PhaseAngle                 = 93.751824414626
    EmissionAngle              = 43.436041872633
    IncidenceAngle             = 51.194520899533
    NorthAzimuth               = 328.91216833605
    OffNadir                   = -0.068262196792972
    SolarLongitude             = 115.53347331512
    LocalTime                  = 14.306934720014
    TargetCenterDistance       = 6.441179974304
    SlantDistance              = 6.2527007047732
    SampleResolution           = 0.42450444081927
    LineResolution             = 0.42450444081927
    PixelResolution            = 0.42450444081927
    MeanGroundResolution       = 0.42920243709832
    SubSolarAzimuth            = 92.619208971733
    SubSolarGroundAzimuth      = 229.08927706091
    SubSolarLatitude           = 2.2167791149862
    SubSolarLongitude          = 85.981654540718
    SubSpacecraftAzimuth       = 266.6386634178
    SubSpacecraftGroundAzimuth = 53.31166995512
    SubSpacecraftLatitude      = 51.079668331088
    SubSpacecraftLongitude     = 182.07154654678
    ParallaxX                  = -0.56570391524198
    ParallaxY                  = 0.7592727247553
    ShadowX                    = 0.81434978037493
    ShadowY                    = -0.93975565736326
    HasLongitudeBoundary       = TRUE
    HasNorthPole               = TRUE
    HasSouthPole               = FALSE
  End_Object
End_Object
End

However, more accurate geometry be achieved using a Bennu shape model provided in the PDS SPICE Archive. These shape models are included in the download configuration. Here are the commands to add the Bennu global shape model:

spiceint from=20190608T005825S208_map_iofL2pan.cub shape=user  model='$osirisrex/kernels/dsk/bennu_g_00880mm_alt_obj_0000n00000_v020.bds'
caminfo from=20190608T005825S208_map_iofL2pan.cub to=20190608T005825S208_map_iofL2pan.shape.pvl

This provides significant differences in the geometry at the image center pixel:

Object = Caminfo
  Object = Parameters
    Program     = caminfo
    IsisVersion = "3.6.1  | 2019-03-14"
    RunDate     = 2020-12-24T00:27:24
    IsisId      = OsirisRex/MapCam/3/0613227461.18586
    From        = 20190608T005825S208_map_iofL2pan.cub
    Lines       = 1024
    Samples     = 1024
    Bands       = 1
  End_Object

  Object = Geometry
    BandsUsed                  = 1
    ReferenceBand              = 1
    OriginalBand               = 1
    Target                     = Bennu
    StartTime                  = 2019-06-08T00:58:25.1174066
    EndTime                    = 2019-06-08T00:58:25.1174066
    CenterLine                 = 512.0
    CenterSample               = 512.0
    CenterLatitude             = 35.086418504259
    CenterLongitude            = 113.89307044494
    CenterRadius               = 237.1296211857
    RightAscension             = 149.07911245571
    Declination                = 73.981202394152
    UpperLeftLongitude         = 184.9904178025
    UpperLeftLatitude          = 1.3951185242827
    LowerLeftLongitude         = NULL
    LowerLeftLatitude          = NULL
    LowerRightLongitude        = NULL
    LowerRightLatitude         = NULL
    UpperRightLongitude        = 342.05371523198
    UpperRightLatitude         = 41.973934677567
    PhaseAngle                 = 93.751824402723
    EmissionAngle              = 49.949588401748
    IncidenceAngle             = 44.61319428432
    NorthAzimuth               = 328.67918931029
    OffNadir                   = -0.38474451979357
    SolarLongitude             = 115.53347331512
    LocalTime                  = 13.860761060282
    TargetCenterDistance       = 6.441179974304
    SlantDistance              = 6.2924667392907
    SampleResolution           = 0.42720421153331
    LineResolution             = 0.42720421153331
    PixelResolution            = 0.42720421153331
    MeanGroundResolution       = 0.43286681459891
    SubSolarAzimuth            = 94.333458422955
    SubSolarGroundAzimuth      = 224.5047216131
    SubSolarLatitude           = 2.2167791149862
    SubSolarLongitude          = 85.981654540718
    SubSpacecraftAzimuth       = 266.63868209054
    SubSpacecraftGroundAzimuth = 49.257236981647
    SubSpacecraftLatitude      = 51.079668331088
    SubSpacecraftLongitude     = 182.07154654678
    ParallaxX                  = -0.77642636079605
    ParallaxY                  = 0.90131734129353
    ShadowX                    = 0.70362754056187
    ShadowY                    = -0.69156684702591
    HasLongitudeBoundary       = TRUE
    HasNorthPole               = TRUE
    HasSouthPole               = FALSE
  End_Object
End_Object
End

Another geometry precision enhancement can be adding by applying the center of gravity structure offsets. This should work in the current system.

The NAIF DSK toolkit containing ray tracing methods is the default when a DSK (.bds) is provided. Using the Bullet ray tracing engine (which OREX/IPWG uses for all geometry) will improve mapping efficiency and provides occlusion detection enhancements. We do not recommend using the Embree ray tracing engine as we have encountered significant problems in our testing. We do not know the state/capabilities of these ray tracing functions in the current version of ISIS.

One important difference with MapCam camera model in UA/ISIS is it uses the OpenCV calibration model to correct for detector distortion. I recommend you process a PolyCam image as well. It uses the old distortion model and may be a better test.

KrisBecker commented 3 years ago

BTW, here is the kernel configuration on the UA/ISIS system for 20190608T005825S208_map_iofL2pan.cub that produced the ellipsoid results:

  Group = Kernels
    NaifFrameCode             = -64361
    LeapSecond                = $base/kernels/lsk/naif0012.tls
    TargetAttitudeShape       = $osirisrex/kernels/pck/bennu_v16.tpc
    TargetPosition            = (Table, $osirisrex/kernels/tspk/de424.bsp,
                                 $osirisrex/kernels/tspk/bennu_refdrmc_v1.bsp,
                                 $osirisrex/kernels/pck/pck00010.tpc)
    InstrumentPointing        = (Table,
                                 $osirisrex/kernels/ck/orx_sc_rel_190603_19060-
                                 9_v02.bc, $osirisrex/kernels/fk/orx_v14.tf)
    Instrument                = ($osirisrex/kernels/ik/orx_ocams_v07.ti,
                                 $osirisrex/kernels/fk/orx_struct_mapcam_v01.b-
                                 c)
    SpacecraftClock           = $osirisrex/kernels/sclk/orx_sclkscet_00061.tsc
    InstrumentPosition        = (Table,
                                 $osirisrex/kernels/spk/orx_190601_190625_1906-
                                 14_od154_v1.bsp)
    InstrumentAddendum        = $osirisrex/kernels/iak/orex_ocams_addendum_v1-
                                0.ti
    ShapeModel                = Null
    InstrumentPositionQuality = Reconstructed
    InstrumentPointingQuality = Reconstructed
    CameraVersion             = 2
  End_Group
End_Object
jlaura commented 3 years ago

@Kelvinrr Is this work completed?

KrisBecker commented 3 years ago

Hi @Kelvinrr, @jessemapel, et al...

Just wanted to let you know that the OREX team has released more data to the PDS, including kernels.

This update provides kernel coverage through 2020-08-17.

It should be a very simply process to add these kernels to ISIS using the kernel maintenance script. Just rerun with the OREX PDS configuration and it will add the new kernels to ISIS and create new versions of the kernel DBs. It should not impact any s/w app tests.

Let me know if you encounter problems.

Thanks...

Kelvinrr commented 3 years ago

@KrisBecker Kernels were updated earlier this week for use internally including the newer kernels you just posted. The kernels will be publicly available with the next ISIS release in April.

KrisBecker commented 3 years ago

I don't know whether to reopen this issue or create a new one, but I am downloading all of the ISIS4+ data area (isisdist.astrogeology.usgs.gov::isisdata) and am seeing some things that may need to be addressed or at least documented.

There is a significant increase in the number of OREX CK and SPK kernels that don't appear to be used or needed in ISIS.

The OLA CKs, of the form orx_ola_YYMMDD_scil2idNNNNN.bc, are instrument specific (i.e., scan mirror adjustments) and there is no support for OLA in ISIS so they are not needed. There are lots of these.

The S/C gimble CKs are also not used. These kernels are of the form orx_sa_rel_YYMMDD_YYMMDD_vXX.bc. These should be removed.

There are also a number of expressly named kernels that I have no idea where they came from. They are not in the PDS archive and the IPWG did not use these kernels. Some examples of these kernels are Equatorial_Stations_1.bc, Plume_Search_1.bc, etc... The point is that these kernels are likely redundant that are also contained in the S/C REL kernels. Anybody know where they came from and their fidelity? Currently, some appear to be tagged as Predict types in the kernel DB file, but I don't think they are necessary.

Currently there are only two types of CK kernels that need to included in the ISIS release - S/C orientation (orx_sc_rel_YYMMDD_YYMMDD_vXX.bc ) and payload structure kernels (orx_struct_mapcam_v01.bc and orx_struct_polycam_v01.bc).

I am also seeing additional SPK kernels that are not in the PDS archive. Some of the examples of these SPK kernels are ORX_Recon_225mSortie_Case02.bsp, OrbitalA_600s_20181203T230000_20190109T000000.bsp, etc... Again, these could be redundant with the standard general coverage kernels (of the form orx_YYMMDD_YYMMDD_YYMMDD_odNNN_vX.bsp) but we have not used them. Anybody know where they came from? Some of these also appear to be tagged as Predict in the DB, which I don't think are needed.

There are s/c SPKs in the TSPK directory that should not be there. In fact, they appear to be a copy what is in the SPK directory. All the kernels in $ISISDATA/osirisrex/kernels/tspk of the form orx_YYMMDD_YYMMDD_YYMMDD_odNNN_vX.bsp should be removed.

There appears to be some residual files that I think are remnants of MacOSX tar file extraction that should be removed. There are some in the IAK directory (e.g., iak/._orex_ocams_addendum_v04.ti).

Unless there are compelling reasons to include these additional kernels, they should be removed so as to try to keep it clean and limit the size of the ISIS DATA download.

Thanks...

jessemapel commented 3 years ago

@KrisBecker I've spun your post off into a new issue so that it's easier for us to track completion on it. I'm also going to get it added to the support prioritization