DOI-USGS / ISIS3

Integrated Software for Imagers and Spectrometers v3. ISIS3 is a digital image processing software package to manipulate imagery collected by current and past NASA and International planetary missions.
https://isis.astrogeology.usgs.gov
Other
200 stars 168 forks source link

Unable to use spkwriter output for Europa #3255

Closed lwellerastro closed 5 years ago

lwellerastro commented 5 years ago

ISIS version(s) affected: isis3.7.0

Description
Spiceinit fails while using spkwriter output for both Galileo SSI and Voyager 1 and 2 images of Europa.

How to reproduce
Create and spacecraft kernel for either/or a Galileo SSI or Voyager image of Europa and try applying it via spiceinit. I had originally done this for jigsaw bundled images, but it also fails on spiceinit-ed images, which is the easier case to reproduce.

Galileo example: Data directory: /work/users/lweller/Isis3Tests/SpkWriter/Galileo_DefaultSpice/

ckwriter from=s0449967752_53_54.cal.cub to=camerakernel.bc summary=camerakernel_summary.doc

spkwriter from=s0449967752_53_54.cal.cub to=spacecraftkernel.bsp summary=spacecraftkernel_summary.doc type=9

cp s0449967752_53_54.cal.cub appynewkernels_s0449967752_53_54.cal.cub

spiceinit from=appynewkernels_s0449967752_53_54.cal.cub ck=camerakernel.bc spk=spacecraftkernel.bsp extra=/usgs/cpkgs/isis3/data/base/kernels/pck/pck00010_msgr_v23.tpc Error = "An unknown NAIF error has been encountered. The short explanation provided by NAIF is [SPICE(SPKINSUFFDATA)]. The Naif error is [Insufficient ephemeris data has been loaded to compute the state of 502 (EUROPA) relative to 0 (SOLAR SYSTEM BARYCENTER) at the ephemeris epoch 1998 MAY 31 21:11:28.735.]"

If I pass spiceinit the camera kernel without the spacecraft kernel, it runs successfully. When spiceinit is passed only the spacecraft kernel it fails in the same way above.

We must explicitly ask for the pck00010 kernel because spiceinit will apply pck00009 by default which contains old/outdated body information for Europa. I ran the same steps above using the default pck00009 kernel and got the same failures, so I don't think that has anything to do with the error.

Voyager example: Data Directory: /work/users/lweller/Isis3Tests/SpkWriter/Voyager_DefaultSpice/

ckwriter from=c2058544.cal.cub to=camerakernel.bc summary=camerakernel_summary.doc

spkwriter from=c2058544.cal.cub to=spacecraftkernel.bsp summary=spacecraftkernel_summary.bsp type=9

cp c2058544.cal.cub applynewkernels_c2058544.cal.cub

spiceinit from=applynewkernels_c2058544.cal.cub ck=camerakernel.bc spk=spacecraftkernel.bsp extra=/usgs/cpkgs/isis3/data/base/kernels/pck/pck00010_msgr_v23.tpc Error = "An unknown NAIF error has been encountered. The short explanation provided by NAIF is [SPICE(SPKINSUFFDATA)]. The Naif error is [Insufficient ephemeris data has been loaded to compute the state of 502 (EUROPA) relative to 0 (SOLAR SYSTEM BARYCENTER) at the ephemeris epoch 1979 JUL 07 11:30:23.118.]"

This one also only fails on the spacecraft kernel.

In both cases of spkwriter I chose type=9 because a note in an old redmine post suggested this was the appropriate type for framers, but the documentation doesn't say. I did try type=13 for the Galileo image and it failed as well.

Possible Solution

Additional context
The reported time in the error message for each case is not the StartTime on the image labels and differs by 48 seconds for the Voyager data and 63 seconds for the Galileo data, which is about the difference between UTC and ET for at the time of data acquisition. This may be the problem, or something in addition to what is going on.

I believe this is the first time we have attempted to create kernels for Voyager and Galileo data.

jessemapel commented 5 years ago

So both Galileo SSI and Voyager Camera use the label UTC StartTime keyword to set the image time. I'm not sure why it's trying to pull the location of Europa at the adjusted StartTime. campt is giving me this image time

(isis3.7.0) [jmapel@astrovm5 Galileo_DefaultSpice]$ campt from=s0449967752_53_54.cal.cub 
campt: Working
Group = GroundPoint
  Filename                   = s0449967752_53_54.cal.cub
  Sample                     = 400.0
  Line                       = 400.0
  PixelValue                 = 0.97459984
  RightAscension             = 347.74502539248 <DEGREE>
  Declination                = -10.456877516855 <DEGREE>
  PlanetocentricLatitude     = 31.16753120247 <DEGREE>
  PlanetographicLatitude     = 31.268396801077 <DEGREE>
  PositiveEast360Longitude   = 216.84431229418 <DEGREE>
  PositiveEast180Longitude   = -143.15568770582 <DEGREE>
  PositiveWest360Longitude   = 143.15568770582 <DEGREE>
  PositiveWest180Longitude   = 143.15568770582 <DEGREE>
  BodyFixedCoordinate        = (-1069.0142357043, -801.01504076463,
                                807.96739329556) <km>
  LocalRadius                = 1561162.3363054 <meters>
  SampleResolution           = 27.820345920464 <meters/pixel>
  LineResolution             = 27.820345920464 <meters/pixel>
  ObliqueDetectorResolution  = 31.237994230144 <meters>
  ObliquePixelResolution     = 31.237994230144 <meters/pix>
  ObliqueLineResolution      = 31.237994230144 <meters>
  ObliqueSampleResolution    = 31.237994230144 <meters>

  # Spacecraft Information
  SpacecraftPosition         = (-3460.3358786123, -2112.5161898026,
                                1061.0994881123) <km>
  SpacecraftAzimuth          = 273.59667515394 <DEGREE>
  SlantDistance              = 2739.0747198998 <km>
  TargetCenterDistance       = 4190.7733377817 <km>
  SubSpacecraftLatitude      = 14.666891772592 <DEGREE>
  SubSpacecraftLongitude     = 211.40383773726 <DEGREE>
  SpacecraftAltitude         = 2628.957812174 <km>
  OffNadirAngle              = 9.8151973680118 <DEGREE>
  SubSpacecraftGroundAzimuth = 198.03112174133 <DEGREE>

  # Sun Information
  SunPosition                = (-660411905.61591, -344856571.58107,
                                21278143.44734) <km>
  SubSolarAzimuth            = 273.93045688571 <DEGREE>
  SolarDistance              = 4.9822407717077 <AU>
  SubSolarLatitude           = 1.6359290724778 <DEGREE>
  SubSolarLongitude          = 207.5728460242 <DEGREE>
  SubSolarGroundAzimuth      = 198.32862960924 <DEGREE>

  # Illumination and Other
  Phase                      = 3.8479397512784 <DEGREE>
  Incidence                  = 30.897597498792 <DEGREE>
  Emission                   = 27.052114250909 <DEGREE>
  NorthAzimuth               = 73.533866141263 <DEGREE>

  # Time
  EphemerisTime              = -50078911.264092 <seconds>
  UTC                        = 1998-05-31T21:10:25.551
  LocalSolarTime             = 12.618097751332 <hour>
  SolarLongitude             = 4.0223032143138 <DEGREE>

  # Look Direction Unit Vectors in Body Fixed, J2000, and Camera Coordinate Systems.
  LookDirectionBodyFixed     = (0.87303994503495, 0.47881174599207,
                                -0.09241518421447)
  LookDirectionJ2000         = (0.96098292238366, -0.208737216207,
                                -0.18149544748318)
  LookDirectionCamera        = (-2.35922392732846e-16, -5.55111512312578e-17,
                                1.0)
End_Group
100% Processed
lwellerastro commented 5 years ago

Cool - so it's an ordering thing then? I never know when to use extra versus something else as it seems to vary based on data set.

jessemapel commented 5 years ago

So I think I found the problem for Galileo at least. The position of Europa is included in the spacecraft position SPK.

(ale) igswzawglt1183:Galileo_DefaultSpice jmapel$ brief -utc -502 /usgs/cpkgs/isis3/data/base/kernels/lsk/naif0012.tls /usgs/cpkgs/isis3/data/base/kernels/spk/de405.bsp 

BRIEF -- Version 4.0.0, September 8, 2010 -- Toolkit Version N0066

Summary for: /usgs/cpkgs/isis3/data/base/kernels/spk/de405.bsp

There is no data:
  for body '502'

(ale) igswzawglt1183:Galileo_DefaultSpice jmapel$ brief -utc -502 /usgs/cpkgs/isis3/data/base/kernels/lsk/naif0012.tls /usgs/cpkgs/isis3/data/galileo/kernels/spk/s000131a.bsp 

BRIEF -- Version 4.0.0, September 8, 2010 -- Toolkit Version N0066

Summary for: /usgs/cpkgs/isis3/data/galileo/kernels/spk/s000131a.bsp

Body: EUROPA (502)
      Start of Interval (UTC)             End of Interval (UTC)
      -----------------------------       -----------------------------
      1997-DEC-01 00:00:00.000            2000-FEB-01 20:00:00.000

(ale) igswzawglt1183:Galileo_DefaultSpice jmapel$ brief -utc -502 /usgs/cpkgs/isis3/data/base/kernels/lsk/naif0012.tls spacecraftkernel.bsp 

BRIEF -- Version 4.0.0, September 8, 2010 -- Toolkit Version N0066

Summary for: spacecraftkernel.bsp

There is no data:
  for body '502'
lwellerastro commented 5 years ago

I have confirmed the work around (extra=spkkernel pck=pck00010) works for Voyager as well but I had to test it on a different file than what was provided above.

jessemapel commented 5 years ago

Current work around is to pass the new kernel as the EXTRA kernel for spiceinit. This way the old kernel is loaded and then the new kernel gets loaded after it. This should result in the state data for Europa being accessible, but the state data for Galileo/Voyager gets overriden by the new kernel data.

This is not an ideal work around and should not be used for production, because the spkwiter kernels cover a significantly smaller range of time than the original kernels and there is a very small chance that we end of reading from the old data. Even though it is highly unlikely, there's is no way for us to identify when this happens.

lwellerastro commented 5 years ago

@jessemapel as for timeline at Europa, I have images that were acquired prior to the 1997-Dec-01 Start Interval for the kernel mentioned above. I pulled all of the Target=Europa images from the PDS and my earliest StartTime is 1996-06-28T01:43:04.667 and my latest is 2002-01-18T15:16:50.319.

jessemapel commented 5 years ago

For reference here's all of the Europa state data available via the Galileo SPKs

(ale) igswzawglt1183:Galileo_DefaultSpice jmapel$ brief -utc -502 /usgs/cpkgs/isis3/data/base/kernels/lsk/naif0012.tls /usgs/cpkgs/isis3/data/galileo/kernels/spk/*.bsp 

BRIEF -- Version 4.0.0, September 8, 2010 -- Toolkit Version N0066

Summary for: /usgs/cpkgs/isis3/data/galileo/kernels/spk/s000131a.bsp

Body: EUROPA (502)
      Start of Interval (UTC)             End of Interval (UTC)
      -----------------------------       -----------------------------
      1997-DEC-01 00:00:00.000            2000-FEB-01 20:00:00.000

Summary for: /usgs/cpkgs/isis3/data/galileo/kernels/spk/s000407a.bsp

Body: EUROPA (502)
      Start of Interval (UTC)             End of Interval (UTC)
      -----------------------------       -----------------------------
      2000-SEP-08 16:38:55.817            2003-OCT-13 00:00:24.817

Summary for: /usgs/cpkgs/isis3/data/galileo/kernels/spk/s000615a.bsp

Body: EUROPA (502)
      Start of Interval (UTC)             End of Interval (UTC)
      -----------------------------       -----------------------------
      2000-JAN-31 23:58:55.815            2003-SEP-21 23:58:55.817

Summary for: /usgs/cpkgs/isis3/data/galileo/kernels/spk/s020128a.bsp

Body: EUROPA (502)
      Start of Interval (UTC)             End of Interval (UTC)
      -----------------------------       -----------------------------
      2000-JAN-31 23:58:55.815            2003-SEP-21 15:19:07.783

Summary for: /usgs/cpkgs/isis3/data/galileo/kernels/spk/s021119a.bsp

Body: EUROPA (502)
      Start of Interval (UTC)             End of Interval (UTC)
      -----------------------------       -----------------------------
      2000-JAN-31 23:58:55.815            2003-SEP-30 11:58:55.817

Summary for: /usgs/cpkgs/isis3/data/galileo/kernels/spk/s030129a.bsp

Body: EUROPA (502)
      Start of Interval (UTC)             End of Interval (UTC)
      -----------------------------       -----------------------------
      2000-JAN-31 23:58:55.815            2003-SEP-30 11:58:55.817

Summary for: /usgs/cpkgs/isis3/data/galileo/kernels/spk/s970311a.bsp

There is no data:
  for body '502'

Summary for: /usgs/cpkgs/isis3/data/galileo/kernels/spk/s980326b.bsp

Body: EUROPA (502)
      Start of Interval (UTC)             End of Interval (UTC)
      -----------------------------       -----------------------------
      1995-SEP-01 23:58:58.817            1995-NOV-20 23:58:58.815
      1995-NOV-20 23:58:58.817            1998-JAN-01 00:00:00.000

Summary for: /usgs/cpkgs/isis3/data/galileo/kernels/spk/s990114a.bsp

Body: EUROPA (502)
      Start of Interval (UTC)             End of Interval (UTC)
      -----------------------------       -----------------------------
      1999-JAN-01 00:00:00.000            2001-MAR-31 23:58:55.814
jessemapel commented 5 years ago

I'm going to go find the documentation for what these kernels are and what they should be used for to help guide what Europa data we should pull out.

jessemapel commented 5 years ago

The current ISIS kernels for Galileo do not match https://naif.jpl.nasa.gov/pub/naif/GLL/kernels/

I don't know if this is intentional or not.

I'm also not seeing any documentation about what kernels we have. It appears to just be loading any of the kernels that cover the image time in alphabetical order.

@scsides @blandoplanet Do either of you have any insight into this?

jessemapel commented 5 years ago

Current contents of the ISIS Data area for Galileo SPKs

igswzawglt1183:ISIS3 jmapel$ ls -l /usgs/cpkgs/isis3/data/galileo/kernels/spk/*.bsp
-rw-r--r-- 1 isis3mgr softlib  4834304 Sep 27  2005 /usgs/cpkgs/isis3/data/galileo/kernels/spk/s000131a.bsp
-rw-r--r-- 1 isis3mgr softlib 10694656 Sep 27  2005 /usgs/cpkgs/isis3/data/galileo/kernels/spk/s000407a.bsp
-rw-r--r-- 1 isis3mgr softlib 10043392 Sep 27  2005 /usgs/cpkgs/isis3/data/galileo/kernels/spk/s000615a.bsp
-rw-r--r-- 1 isis3mgr softlib 12417024 Sep 27  2005 /usgs/cpkgs/isis3/data/galileo/kernels/spk/s020128a.bsp
-rw-r--r-- 1 isis3mgr softlib 12127232 Sep 27  2005 /usgs/cpkgs/isis3/data/galileo/kernels/spk/s021119a.bsp
-rw-r--r-- 1 isis3mgr softlib 12137472 Sep 27  2005 /usgs/cpkgs/isis3/data/galileo/kernels/spk/s030129a.bsp
-rw-r--r-- 1 isis3mgr softlib  1496064 Sep 27  2005 /usgs/cpkgs/isis3/data/galileo/kernels/spk/s970311a.bsp
-rw-r--r-- 1 isis3mgr softlib  7127040 Sep 27  2005 /usgs/cpkgs/isis3/data/galileo/kernels/spk/s980326b.bsp
-rw-r--r-- 1 isis3mgr softlib  5399552 Sep 27  2005 /usgs/cpkgs/isis3/data/galileo/kernels/spk/s990114a.bsp

Current kernel.db

Object = SpacecraftPosition
  RunTime = 2005-09-27T16:06:19

  Group = Dependencies
    LeapsecondKernel = $base/kernels/lsk/naif0008.tls
  End_Group

  Group = Selection
    Time = ("1997 DEC 01 00:01:03.183 TDB", "2000 FEB 01 20:01:04.184 TDB")
    File = $galileo/kernels/spk/s000131a.bsp
    Type = Reconstructed
  End_Group

  Group = Selection
    Time = ("2000 SEP 08 16:40:00.000 TDB", "2003 OCT 13 00:01:29.000 TDB")
    File = $galileo/kernels/spk/s000407a.bsp
    Type = Reconstructed
  End_Group

  Group = Selection
    Time = ("2000 FEB 01 00:00:00.000 TDB", "2003 SEP 22 00:00:00.000 TDB")
    File = $galileo/kernels/spk/s000615a.bsp
    Type = Reconstructed
  End_Group

  Group = Selection
    Time = ("2000 FEB 01 00:00:00.000 TDB", "2003 SEP 21 15:20:11.966 TDB")
    File = $galileo/kernels/spk/s020128a.bsp
    Type = Reconstructed
  End_Group

  Group = Selection
    Time = ("2000 FEB 01 00:00:00.000 TDB", "2003 SEP 30 12:00:00.000 TDB")
    File = $galileo/kernels/spk/s021119a.bsp
    Type = Reconstructed
  End_Group

  Group = Selection
    Time = ("2000 FEB 01 00:00:00.000 TDB", "2003 SEP 30 12:00:00.000 TDB")
    File = $galileo/kernels/spk/s030129a.bsp
    Type = Reconstructed
  End_Group

  Group = Selection
    Time = ("1989 OCT 19 01:29:33.260 TDB", "1990 SEP 05 00:00:57.182 TDB")
    Time = ("1990 SEP 05 00:00:57.184 TDB", "1992 JAN 01 00:00:00.000 TDB")
    Time = ("1992 JAN 01 00:00:00.000 TDB", "1995 JUL 02 00:01:01.184 TDB")
    File = $galileo/kernels/spk/s970311a.bsp
    Type = Reconstructed
  End_Group

  Group = Selection
    Time = ("1995 JUL 01 00:00:00.000 TDB", "1995 NOV 20 23:59:59.998 TDB")
    Time = ("1995 NOV 21 00:00:00.000 TDB", "1998 JAN 01 00:01:03.183 TDB")
    File = $galileo/kernels/spk/s980326b.bsp
    Type = Reconstructed
  End_Group

  Group = Selection
    Time = ("1999 JAN 01 00:01:04.183 TDB", "2001 APR 01 00:00:00.000 TDB")
    File = $galileo/kernels/spk/s990114a.bsp
    Type = Reconstructed
  End_Group
End_Object
End
jessemapel commented 5 years ago

NAIF has a collection of spks for satellites similar to the spk we use for the main solor system bodies

https://naif.jpl.nasa.gov/pub/naif/generic_kernels/spk/satellites/

Is it safe to use this with the spkwriter output? We would probably need to redo all of the processing with it :/

lwellerastro commented 5 years ago

@jessemapel Spiced Galileo images are here: /work/users/lweller/Europa/Galileo/Cal/*cub 482 images

Deltack'd Voyager images are here: /work/users/lweller/Europa/VGR/Resection/Cal/*cub 241 images

jessemapel commented 5 years ago

Directly comparing the position of Europa relative to Jupiter between the Galileo SPKs and the NAIF satellite SPK, I'm seeing anywhere from 2km to 25km differences.

https://gist.github.com/jessemapel/c47b4a563afe0ce8b448be3075f7f49a

jessemapel commented 5 years ago

We've decided we're going to stage the satellites kernel for Galileo and Voyager so that it always uses that for Europa's location. We'll also update to de430.bsp for the planets kernel.

jessemapel commented 5 years ago

This is for my reference. Updating to de430 has caused a good number of test failures. A lot are due to a new way Mars is included in de430. Others are due to expected changes in illumination.

The following tests FAILED:
    115 - isis3_unit_test_JP2Importer (Failed)
    333 - isis3_unit_test_KernelDb (Failed)
    668 - isis2std_app_test_jpeg2000 (Failed)
    891 - shadow_app_test_moon (Failed)
    916 - spiceinit_app_test_allowCkReconAndSmith (Failed)
    917 - spiceinit_app_test_ckConfigFile (Failed)
    918 - spiceinit_app_test_default (Failed)
    921 - spiceinit_app_test_padding (Failed)
    923 - spiceserver_app_test_cubetest (Failed)
    924 - spiceserver_app_test_simplerequest (Failed)
    933 - std2isis_app_test_jpeg2000 (Failed)
    1179 - clementine_module_test_clementine (Failed)
    1318 - mical_app_test_case01 (Failed)
    1319 - mical_app_test_case02 (Failed)
    1330 - mdiscal_app_test_contamination (Failed)
    1332 - mdiscal_app_test_default (Failed)
    1341 - mdisproc_app_test_cdr (Failed)
    1342 - mdisproc_app_test_default (Failed)
    1343 - mdisproc_app_test_trim (Failed)
    1356 - moccal_app_test_default (Failed)
    1358 - moccal_app_test_nullwago (Failed)
    1362 - mocproc_app_test_case01 (Failed)
    1363 - mocproc_app_test_case02 (Failed)
    1364 - mocproc_app_test_case03 (Failed)
    1366 - mgs_module_test_moc (Failed)
    1377 - ctxcal_app_test_default (Failed)
    1382 - hical_app_test_default (Failed)
    1385 - hicalbeta_app_test_default (Failed)
    1388 - hicalproc_app_test_default (Failed)
    1389 - hicalproc_app_test_destripe (Failed)
    1390 - hicalproc_app_test_furrows (Failed)
    1425 - hijitter_app_test_default (Failed)
    1426 - hijitter_app_test_withCrop (Failed)
    1453 - mro_module_test_hirise (Failed)
    1486 - thmbasemap1_app_test_dayonly (Failed)
    1487 - thmbasemap1_app_test_nightonly (Failed)
    1492 - thmproc_app_test_ir (Failed)
    1493 - thmproc_app_test_vis (Failed)
Errors while running CTest

The following are due to not building with JP2K

    115 - isis3_unit_test_JP2Importer (Failed)
    668 - isis2std_app_test_jpeg2000 (Failed)
    933 - std2isis_app_test_jpeg2000 (Failed)
    933 - std2isis_app_test_jpeg2000 (Failed)
jessemapel commented 5 years ago

The Galileo data area has been setup for this to work now. Voyager1 and Voyager2 are currently waiting for after the 3.7.1 release. I will then pull down the new spice from #1242.

jessemapel commented 5 years ago

SPICE is pulled down and now in the data area.

lwellerastro commented 5 years ago

@jessemapel, will I have to do anything special when applying spkwriter output kernels to the Voyager or Galileo data I am working with? I am having a hard remembering or locating details to answer this question. We did have a work around at some point to deal with how and when spacecraft information was/is being loaded for Galileo in particular (I think), but I don't think that was an issue for Voyager. Oh, it's in this thread (the work around):

Current work around is to pass the new kernel as the EXTRA kernel for spiceinit. This way the old kernel is loaded and then the new kernel gets loaded after it. This should result in the state data for Europa being accessible, but the state data for Galileo/Voyager gets overriden by the new kernel data.

This is not an ideal work around and should not be used for production, because the spkwiter kernels cover a significantly smaller range of time than the original kernels and there is a very small chance that we end of reading from the old data. Even though it is highly unlikely, there's is no way for us to identify when this happens.

Could you summarize here (or under an existing astro discuss post) if a user has to do something special to use spkwriter generated kernels for these missions in particular? I intend on setting pck=/usgs/cpkgs/isis3/data/base/kernels/pck/pck00010_msgr_v23.tpc (which has been tested and works fine), but I'm not clear on whether or not these new Voyager and the recently updated Galileo kernels still require me to use the spiceinit extra parameter when using an spkwriter generated kernel. Thanks!

jessemapel commented 5 years ago

@lwellerastro You should still use the extra parameter to override the PCK. You do not need to do anything special to use spkwriter kernels with the updated data. The ephemerides for Europa have changed a little bit with the new kernels, so it would be best to reprocess and then recreate your new SPK.

lwellerastro commented 5 years ago

Thanks @jessemapel, that's what I was hoping to hear!