search

DOI-USGS / ISIS3

Integrated Software for Imagers and Spectrometers v3. ISIS3 is a digital image processing software package to manipulate imagery collected by current and past NASA and International planetary missions.
https://isis.astrogeology.usgs.gov
Other
197 stars 167 forks source link

State of ISIS Test Data #4744

Closed KrisBecker closed 2 years ago

KrisBecker commented 2 years ago

ISIS version(s) affected: 6.0.0+

Description
Not sure if this is an issue but thought I would let you know results of running ISIS tests on a recent build as described in #4742. Here is the report:

99% tests passed, 18 tests failed out of 2348

Label Time Summary:
apollo          =   3.67 sec*proc (4 tests)
app             = 4728.26 sec*proc (796 tests)
cassini         =   3.44 sec*proc (5 tests)
chandrayaan1    =   1.21 sec*proc (1 test)
clementine      =   1.21 sec*proc (4 tests)
dawn            =   0.71 sec*proc (2 tests)
galileo         =   0.31 sec*proc (1 test)
hayabusa        =   0.94 sec*proc (2 tests)
hayabusa2       =   0.32 sec*proc (1 test)
isis            = 148.22 sec*proc (322 tests)
juno            =   0.27 sec*proc (1 test)
kaguya          =   1.00 sec*proc (2 tests)
lo              =   0.95 sec*proc (3 tests)
lro             =   1.46 sec*proc (3 tests)
mariner         =   1.18 sec*proc (1 test)
messenger       =   0.40 sec*proc (1 test)
mex             =   0.70 sec*proc (2 tests)
mgs             =   2.43 sec*proc (4 tests)
mro             =   3.72 sec*proc (6 tests)
near            =   0.28 sec*proc (1 test)
newhorizons     =   1.22 sec*proc (4 tests)
odyssey         =   0.60 sec*proc (2 tests)
rosetta         =   1.31 sec*proc (2 tests)
tgo             =   0.33 sec*proc (1 test)
unit            = 177.30 sec*proc (377 tests)
viking          =   0.66 sec*proc (1 test)
voyager         =   0.76 sec*proc (1 test)

Total Test time (real) = 6469.14 sec

The following tests did not run:
    1698 - TempTestingFiles.FunctionalTestJitterfitDefault (Disabled)
    1716 - TempTestingFiles.UnitTestImageImporterTestJpeg (Disabled)

The following tests FAILED:
     44 - isis_unit_test_Cube (Failed)
    109 - isis_unit_test_JP2Importer (Failed)
    438 - edrget_app_test_connectErrors (Failed)
    439 - edrget_app_test_ftp (Failed)
    440 - edrget_app_test_http (Failed)
    441 - edrget_app_test_timeout (Failed)
    541 - isis2std_app_test_jpeg2000 (Failed)
    1035 - mdisedrinfo_app_test_kernelchk (Failed)
    1148 - tgocassismos_app_test_default (Failed)
    1151 - tgo_module_test_coloredMosaic (Failed)
    1152 - tgo_module_test_ingestReingest (Failed)
    1153 - tgo_module_test_mapProjectedReingested (Failed)
    1154 - tgo_module_test_projSingleStichedFrame (Failed)
    1155 - tgo_module_test_singleColorMosaicReingest (Failed)
    1156 - tgo_module_test_uncontrolledSingleColorMosaic (Failed)
    1712 - TempTestingFiles.FunctionalTestStd2isisJp2 (Failed)
    1718 - TempTestingFiles.UnitTestImageImporterStd2IsisJp2 (Failed)
    1815 - MroHiriseCube.FunctionalTestHimosDefault (Failed)
Errors while running CTest
Output from these tests are in: /Users/kbecker/ISIS/Isis4Dev/ErosPDART/ISIS3/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.

This seems to be very similar to the reported state as a result of issue #4661.

What concerns me is that it appears as though no new test data exists since 2021-08-04. I confirmed this using the command rclone lsl remote:asc-isisdata/isis_testData > testdata_listing.lis and then sorting using sort -s -r -k2 testdata_listing.lis. This sorts by date in the second column with newest first. This produces the following (only a few lines are shown):

(base36) kbmacusb[1638]: sort -s -r   -k2 testdata_listing.lis
  2064384 2021-08-24 07:17:50.155866765 isis/src/tgo/apps/tgocassisrdrgen/tsts/raw/truth/CAS-MCO-2016-11-22T15.45.50.984-BLU-03000-B1_modifiedId.img
  2064384 2021-08-24 07:17:50.086867403 isis/src/tgo/apps/tgocassisrdrgen/tsts/raw/truth/CAS-MCO-2016-11-22T15.45.50.984-BLU-03000-B1.img
     5275 2021-08-24 07:17:50.020868013 isis/src/tgo/apps/tgocassisrdrgen/tsts/raw/truth/CAS-MCO-2016-11-22T15.45.50.984-BLU-03000-B1.xmlLabel.txt
     5282 2021-08-24 07:17:49.963868540 isis/src/tgo/apps/tgocassisrdrgen/tsts/raw/truth/CAS-MCO-2016-11-22T15.45.50.984-BLU-03000-B1_modifiedId.xmlLabel.txt
  2130630 2021-08-24 07:17:49.658871358 isis/src/tgo/apps/tgocassisrdrgen/tsts/raw/truth/CAS-MCO-2016-11-22T15.45.50.984-BLU-03000-B1.cub
      463 2021-08-24 07:16:56.196365526 isis/src/system/apps/kerneldbgen/tsts/coverageLevel/truth/kernel_segment.db.pvl
     4129 2021-08-24 07:16:56.141366034 isis/src/system/apps/kerneldbgen/tsts/coverageLevel/truth/kernel_interval.db.pvl
     1468 2021-08-20 07:40:58.000000000 isis/src/tgo/apps/tgocassis2isis/tsts/PSA_label/truth/MY36_015782_024_0_PAN.lbl
157845680 2021-08-18 12:33:42.000000000 isis/src/tgo/apps/tgocassis2isis/tsts/PSA_label/input/MY36_015782_024_0_PAN.dat
    27786 2021-08-18 12:33:37.000000000 isis/src/tgo/apps/tgocassis2isis/tsts/PSA_label/input/MY36_015782_024_0_PAN.xml
       54 2021-06-11 00:51:44.000000000 isis/src/mro/tsts/hirise/truth.Darwin.x86_64.MacOSX10_14/PSP_010186_1785_RED3_1.IMG--getsnOutput.txt
    15930 2021-06-11 00:51:44.000000000 isis/src/mro/tsts/hirise/truth.Darwin.x86_64.MacOSX10_14/PSP_010186_1785_RED3_1.IMG--finalOutput.pvl
      835 2021-06-11 00:20:43.000000000 isis/src/mro/apps/hijitter/tsts/withCrop/truth.Darwin.x86_64.MacOSX10_14/red6.stats.pvl
      838 2021-06-11 00:20:43.000000000 isis/src/mro/apps/hijitter/tsts/withCrop/truth.Darwin.x86_64.MacOSX10_14/red5.stats.pvl
      836 2021-06-11 00:20:43.000000000 isis/src/mro/apps/hijitter/tsts/withCrop/truth.Darwin.x86_64.MacOSX10_14/red4.stats.pvl
   143706 2021-06-11 00:20:43.000000000 isis/src/mro/apps/hijitter/tsts/withCrop/truth.Darwin.x86_64.MacOSX10_14/cktable.csv

It seems a bit odd that the last update to test data was August 2021.

Can you please confirm that the ISIS test data is up-to-date?

How to reproduce

Possible Solution

Additional context

jessemapel commented 2 years ago

This is concerning. We've got some work coming up to modify some of the tests as we have new runners for CI. It may be a good time to add a Jenkins workflow that auto-syncs test data changes.

jessemapel commented 2 years ago

I've manually updated the test data, you should be able to pull it and see most of these resolve. These tests will likely still fail as they require Kakadu.

    109 - isis_unit_test_JP2Importer (Failed)
    541 - isis2std_app_test_jpeg2000 (Failed)
    1712 - TempTestingFiles.FunctionalTestStd2isisJp2 (Failed)
    1718 - TempTestingFiles.UnitTestImageImporterStd2IsisJp2 (Failed)
jessemapel commented 2 years ago

I've also added a cron job to sync the local test data to s3 each morning. I'll close this tomorrow after I can confirm it runs successfully

KrisBecker commented 2 years ago

I updated my test data area today so I thought I would post differences:

(base) [1030]: diff testdata_listing_20220111_sorted.lis testdata_listing_20220208_sorted.lis
0a1,48
>       180 2022-01-14 10:15:43.215895889 isis/src/mro/apps/hijitter/tsts/default/input/red4.stats.pvl.DIFF~
>       238 2022-01-14 10:15:42.979863613 isis/src/mro/apps/hijitter/tsts/default/input/red6.stats.pvl.DIFF
>       176 2022-01-14 10:15:42.739830789 isis/src/mro/apps/hijitter/tsts/default/input/red4.stats.pvl.DIFF
>  16245124 2022-01-14 10:15:42.485796051 isis/src/mro/apps/hijitter/tsts/default/input/PSP_007556_2010_RED6.balance.cropped.cub
>  16597410 2022-01-14 10:15:11.898613365 isis/src/mro/apps/hijitter/tsts/default/input/PSP_007556_2010_RED5.balance.cropped.cub
>      2227 2022-01-14 10:14:28.280648682 isis/src/mro/apps/hijitter/tsts/default/input/PSP_007556_2010.cropped.txt
>        81 2022-01-14 10:14:28.029614354 isis/src/mro/apps/hijitter/tsts/default/input/cktable.csv.DIFF
>  16256391 2022-01-14 10:14:27.794582214 isis/src/mro/apps/hijitter/tsts/default/input/PSP_007556_2010_RED4.balance.cropped.cub
>       311 2022-01-14 10:14:14.402750884 isis/src/mro/apps/hijitter/tsts/default/input/hijitreg.p0745.s1570.def
>       241 2022-01-14 10:14:14.147716007 isis/src/mro/apps/hijitter/tsts/default/input/red5.stats.pvl.DIFF
>       241 2022-01-14 10:14:13.854675934 isis/src/mro/apps/hijitter/tsts/default/input/red5.stats.pvl.DIFF~
>       224 2022-01-14 10:14:13.611642700 isis/src/mro/apps/hijitter/tsts/default/input/red6.stats.pvl.DIFF~
>     70037 2022-01-14 10:14:13.360608370 isis/src/mro/apps/hijitter/tsts/default/input/PSP_007556_2010.txt
>    143706 2022-01-14 10:12:13.420206117 isis/src/mro/apps/hijitter/tsts/default/truth.Darwin.x86_64.MacOSX11_6/cktable.csv
>       835 2022-01-14 10:12:12.896134447 isis/src/mro/apps/hijitter/tsts/default/truth.Darwin.x86_64.MacOSX11_6/red6.stats.pvl
>  74909314 2022-01-14 10:12:12.644099979 isis/src/mro/apps/hijitter/tsts/default/truth.Darwin.x86_64.MacOSX11_6/PSP_007556_2010_RED4.balance.cropped.cub
>       837 2022-01-14 10:09:11.753361085 isis/src/mro/apps/hijitter/tsts/default/truth.Darwin.x86_64.MacOSX11_6/red5.stats.pvl
>       835 2022-01-14 10:09:11.456320460 isis/src/mro/apps/hijitter/tsts/default/truth.Darwin.x86_64.MacOSX11_6/red4.stats.pvl
>       174 2021-12-03 11:17:07.585231342 isis/src/tgo/apps/tgocassismos/tsts/default/input/tgocassismos.pvl.DIFF
>  20015733 2021-12-03 11:17:07.372202349 isis/src/tgo/apps/tgocassismos/tsts/default/input/CAS-MCO-2016-11-26T22.32.15.582-RED-01001-B1_equi.cub
>  20087613 2021-12-03 11:16:51.751076363 isis/src/tgo/apps/tgocassismos/tsts/default/input/CAS-MCO-2016-11-26T22.32.14.582-RED-01000-B1_equi.cub
>  18698981 2021-12-03 11:16:35.937924238 isis/src/tgo/apps/tgocassismos/tsts/default/input/CAS-MCO-2016-11-26T22.32.18.582-RED-01004-B1_equi.cub
>  19483253 2021-12-03 11:16:20.114770745 isis/src/tgo/apps/tgocassismos/tsts/default/input/CAS-MCO-2016-11-26T22.32.17.582-RED-01003-B1_equi.cub
>  19636853 2021-12-03 11:16:03.695536123 isis/src/tgo/apps/tgocassismos/tsts/default/input/CAS-MCO-2016-11-26T22.32.16.582-RED-01002-B1_equi.cub
>  54468026 2021-12-03 11:07:00.777642849 isis/src/tgo/apps/tgocassismos/tsts/default/truth/tgocassismos_tracking.cub
>      4817 2021-12-03 11:06:22.993500026 isis/src/tgo/apps/tgocassismos/tsts/default/truth/tgocassismos.pvl
>  54469497 2021-12-03 11:06:22.801473890 isis/src/tgo/apps/tgocassismos/tsts/default/truth/tgocassismos.cub
>       862 2021-12-03 11:05:38.577454487 isis/src/tgo/apps/tgocassismos/tsts/default/truth/tgocassismos_tracking.pvl
>        32 2021-11-03 13:55:19.153806704 isis/src/tgo/tsts/coloredMosaic/truth.Darwin.x86_64.MacOSX10_13/truth/message.txt
>        32 2021-11-03 13:55:16.877954184 isis/src/tgo/tsts/coloredMosaic/truth.Darwin.x86_64.MacOSX10_15/truth/message.txt
>        32 2021-11-03 13:55:03.520819681 isis/src/tgo/tsts/coloredMosaic/truth/message.txt
>        32 2021-11-03 13:52:37.843251644 isis/src/tgo/tsts/ingestReingest/truth/message.txt
>        32 2021-11-03 13:51:01.942453162 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/truth/message.txt
>        32 2021-11-03 13:50:59.989579381 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_13/truth/message.txt
>        32 2021-11-03 13:50:52.133087146 isis/src/tgo/tsts/mapProjectedReingested/truth/message.txt
>        32 2021-11-03 13:44:33.467509046 isis/src/tgo/tsts/projSingleStichedFrame/truth.Darwin.x86_64.MacOSX10_13/truth/message.txt
>        32 2021-11-03 13:44:31.206654548 isis/src/tgo/tsts/projSingleStichedFrame/truth.Darwin.x86_64.MacOSX10_15/truth/message.txt
>        32 2021-11-03 13:44:07.934152079 isis/src/tgo/tsts/projSingleStichedFrame/truth/message.txt
>        32 2021-11-03 13:32:19.236890650 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_13/output/message.txt
>        32 2021-11-03 13:32:16.647077170 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_15/output/message.txt
>        32 2021-11-03 13:31:26.679677321 isis/src/tgo/tsts/singleColorMosaicReingest/truth/message.txt
>        32 2021-11-03 13:01:49.829077502 isis/src/tgo/tsts/uncontrolledSingleColorMosaic/truth.Darwin.x86_64.MacOSX10_13/output/message.txt
>        32 2021-11-03 13:01:47.243244214 isis/src/tgo/tsts/uncontrolledSingleColorMosaic/truth.Darwin.x86_64.MacOSX10_15/output/message.txt
>        32 2021-11-03 13:00:31.606120486 isis/src/tgo/tsts/uncontrolledSingleColorMosaic/truth/message.txt
>        32 2021-11-03 12:57:13.000000000 isis/src/base/apps/edrget/tsts/ftp/truth/message.txt
>        32 2021-11-03 12:10:33.112559854 isis/src/base/apps/edrget/tsts/timeout/truth/message.txt
>        32 2021-11-03 11:54:58.431313354 isis/src/base/apps/edrget/tsts/http/truth/message.txt
>        32 2021-11-03 11:50:01.634406261 isis/src/base/apps/edrget/tsts/connectErrors/truth/message.txt
53,68d100
<     11768 2021-06-03 11:01:53.664908786 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_13/CAS-M01-2018-05-05T23.11.mosaic.xmlLabel.txt
<      5540 2021-06-03 11:01:09.101049183 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_13/CAS-M01-2018-05-05T23.11.mosaic.pvl
<    168992 2021-06-03 11:00:29.238385715 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_13/CAS-M01-2018-05-05T23.11.mosaic.cub
<     12289 2021-06-03 10:44:18.429974640 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.mosaic.xmlLabel.txt
<      5543 2021-06-03 10:43:10.129134084 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.mosaic.pvl
<     48032 2021-06-03 10:42:21.394107273 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.mosaic.img
<    116615 2021-06-03 10:40:02.735340793 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.mosaic.cub
<      1305 2021-06-03 08:49:01.545764651 isis/src/tgo/tsts/uncontrolledSingleColorMosaic/truth/equi.pvl
<      5523 2021-06-03 08:49:01.485768566 isis/src/tgo/tsts/uncontrolledSingleColorMosaic/truth/cassisMosaic.pvl
<    352336 2021-06-03 08:49:01.421772741 isis/src/tgo/tsts/uncontrolledSingleColorMosaic/truth/cassisMosaic.img
<     12279 2021-06-03 08:49:01.361776656 isis/src/tgo/tsts/uncontrolledSingleColorMosaic/truth/cassisMosaic.xmlLabel.txt
<    420915 2021-06-03 08:49:00.816812214 isis/src/tgo/tsts/uncontrolledSingleColorMosaic/truth/cassisMosaic.cub
<      5542 2021-06-03 08:41:46.937145007 isis/src/tgo/tsts/singleColorMosaicReingest/truth/CAS-M01-2018-05-05T23.11.mosaic.pvl
<     48032 2021-06-03 08:41:46.874149124 isis/src/tgo/tsts/singleColorMosaicReingest/truth/CAS-M01-2018-05-05T23.11.mosaic.img
<     12288 2021-06-03 08:41:46.815152979 isis/src/tgo/tsts/singleColorMosaicReingest/truth/CAS-M01-2018-05-05T23.11.mosaic.xmlLabel.txt
<    116611 2021-06-03 08:41:46.150196436 isis/src/tgo/tsts/singleColorMosaicReingest/truth/CAS-M01-2018-05-05T23.11.mosaic.cub
149d180
<       174 2021-06-01 13:58:27.958028899 isis/src/tgo/apps/tgocassismos/tsts/default/input/tgocassismos.pvl.DIFF
168,175d198
<     14105 2021-04-27 15:08:32.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.pvl
<      2134 2021-04-27 15:08:32.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.reingested.pvl
<      7963 2021-04-27 15:08:31.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.xmlLabel.txt
<     14705 2021-04-27 15:08:31.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.pvl
<      2560 2021-04-27 15:08:31.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.img
<     77551 2021-04-27 15:08:30.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.reingested.cub
<     70611 2021-04-27 15:08:28.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.cub
<   1919758 2021-04-27 15:08:25.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.cub
189,196c212,223
<     14105 2021-03-25 21:44:29.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.pvl
<      7963 2021-03-25 21:44:29.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.xmlLabel.txt
<      2134 2021-03-25 21:44:29.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.reingested.pvl
<     77545 2021-03-25 21:44:29.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.reingested.cub
<     14705 2021-03-25 21:44:29.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.pvl
<      2560 2021-03-25 21:44:29.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.img
<     70593 2021-03-25 21:44:29.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.cub
<   1919740 2021-03-25 21:44:29.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.cub
---
>     14105 2021-03-25 21:44:29.735433362 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.pvl
>      7963 2021-03-25 21:44:29.732433556 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.xmlLabel.txt
>      2134 2021-03-25 21:44:29.729433751 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.reingested.pvl
>     77545 2021-03-25 21:44:29.726433945 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.reingested.cub
>     14705 2021-03-25 21:44:29.723434139 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.pvl
>      2560 2021-03-25 21:44:29.721434269 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.img
>     70593 2021-03-25 21:44:29.683436731 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.cub
>   1919740 2021-03-25 21:44:29.638439646 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.cub
>     12289 2021-03-25 21:43:53.823759727 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.mosaic.xmlLabel.txt
>      5543 2021-03-25 21:43:53.811760504 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.mosaic.pvl
>     48032 2021-03-25 21:43:53.795761541 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.mosaic.img
>    116521 2021-03-25 21:43:53.784762253 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_15/CAS-M01-2018-05-05T23.11.mosaic.cub
215d241
<      5083 2021-03-25 14:44:32.024514830 isis/src/tgo/tsts/ingestReingest/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.xmlLabel.txt
223a250
>     11769 2021-03-25 13:55:47.550848706 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_13/CAS-M01-2018-05-05T23.11.mosaic.xmlLabel.txt
226,227c253
<      7760 2021-03-25 13:47:17.606218436 isis/src/tgo/tsts/projSingleStichedFrame/truth/exported.xmlLabel.txt
<      7960 2021-03-25 13:45:19.000000000 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_13/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.xmlLabel.txt
---
>      7960 2021-03-25 13:45:19.571422163 isis/src/tgo/tsts/mapProjectedReingested/truth.Darwin.x86_64.MacOSX10_13/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.equi.xmlLabel.txt
229d254
<      6714 2021-03-25 13:34:16.590518068 isis/src/tgo/tsts/coloredMosaic/truth/coloredMosaic.xmlLabel.txt
611,613d635
<      1303 2020-10-03 11:12:42.849155132 isis/src/tgo/tsts/coloredMosaic/truth/equi.pvl
<   6805952 2020-10-03 11:12:42.762149424 isis/src/tgo/tsts/coloredMosaic/truth/coloredMosaic.img
<   7183697 2020-10-03 11:12:42.170110576 isis/src/tgo/tsts/coloredMosaic/truth/coloredMosaic.cub
627,631d648
<      1452 2020-10-02 20:56:18.460056264 isis/src/tgo/tsts/ingestReingest/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.reingested.pvl
<      4831 2020-10-02 20:56:18.408052838 isis/src/tgo/tsts/ingestReingest/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.pvl
<   1835008 2020-10-02 20:56:18.348048884 isis/src/tgo/tsts/ingestReingest/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.img
<   1906589 2020-10-02 20:56:17.728008020 isis/src/tgo/tsts/ingestReingest/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.reingested.cub
<   1917336 2020-10-02 20:56:17.272978032 isis/src/tgo/tsts/ingestReingest/truth/CAS-M01-2018-05-05T23.11.48.767-RED-01029-B1.cub
764,771d780
<      1292 2020-08-21 13:10:16.000000000 isis/src/tgo/tsts/projSingleStichedFrame/truth/stitched.pvl
<  17985456 2020-08-21 13:10:16.000000000 isis/src/tgo/tsts/projSingleStichedFrame/truth/exported.img
<  18633552 2020-08-21 13:09:57.000000000 isis/src/tgo/tsts/projSingleStichedFrame/truth/projected.cub
<  16844431 2020-08-21 13:09:39.000000000 isis/src/tgo/tsts/projSingleStichedFrame/truth/stitched-2016-11-26T22:50:27.381.cub
<   2178317 2020-08-21 13:09:38.000000000 isis/src/tgo/tsts/projSingleStichedFrame/truth/nirframelet.cub
<    136968 2020-08-21 13:09:33.000000000 isis/src/tgo/tsts/projSingleStichedFrame/truth/bluframelet.cub
<   2178320 2020-08-21 13:09:26.000000000 isis/src/tgo/tsts/projSingleStichedFrame/truth/redframelet.cub
<   2374928 2020-08-21 13:09:15.000000000 isis/src/tgo/tsts/projSingleStichedFrame/truth/panframelet.cub
832a842
>      5540 2020-07-17 14:42:47.000000000 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_13/CAS-M01-2018-05-05T23.11.mosaic.pvl
833a844
>    116489 2020-07-17 14:42:45.000000000 isis/src/tgo/tsts/singleColorMosaicReingest/truth.Darwin.x86_64.MacOSX10_13/CAS-M01-2018-05-05T23.11.mosaic.cub
1094,1097d1104
<       862 2019-08-15 18:20:28.000000000 isis/src/tgo/apps/tgocassismos/tsts/default/truth/tgocassismos_tracking.pvl
<  54468026 2019-08-15 18:20:28.000000000 isis/src/tgo/apps/tgocassismos/tsts/default/truth/tgocassismos_tracking.cub
<      4801 2019-08-15 18:20:28.000000000 isis/src/tgo/apps/tgocassismos/tsts/default/truth/tgocassismos.pvl
<  54469505 2019-08-15 18:20:28.000000000 isis/src/tgo/apps/tgocassismos/tsts/default/truth/tgocassismos.cub
1150d1156
<       241 2019-05-16 17:23:12.000000000 isis/src/mro/apps/hijitter/tsts/default/input/red5.stats.pvl.DIFF
1424,1425d1429
<        43 2018-10-17 15:52:38.216141132 isis/src/base/apps/edrget/tsts/timeout/truth/httperror.txt
<        43 2018-10-17 15:52:38.195139187 isis/src/base/apps/edrget/tsts/timeout/truth/ftperror.txt
1495,1499d1498
<  20015733 2018-10-03 19:09:40.000000000 isis/src/tgo/apps/tgocassismos/tsts/default/input/CAS-MCO-2016-11-26T22.32.15.582-RED-01001-B1_equi.cub
<  19483253 2018-10-03 19:09:18.000000000 isis/src/tgo/apps/tgocassismos/tsts/default/input/CAS-MCO-2016-11-26T22.32.17.582-RED-01003-B1_equi.cub
<  18698981 2018-10-03 19:09:05.000000000 isis/src/tgo/apps/tgocassismos/tsts/default/input/CAS-MCO-2016-11-26T22.32.18.582-RED-01004-B1_equi.cub
<  20087613 2018-10-03 19:08:06.000000000 isis/src/tgo/apps/tgocassismos/tsts/default/input/CAS-MCO-2016-11-26T22.32.14.582-RED-01000-B1_equi.cub
<  19636853 2018-10-03 19:07:37.000000000 isis/src/tgo/apps/tgocassismos/tsts/default/input/CAS-MCO-2016-11-26T22.32.16.582-RED-01002-B1_equi.cub
2166,2174d2164
<        61 2017-03-02 10:44:45.474094509 isis/src/base/apps/edrget/tsts/connectErrors/truth/noScheme.txt
<       236 2017-03-02 10:44:45.445093707 isis/src/base/apps/edrget/tsts/connectErrors/truth/httpFileDNE.txt
<       234 2017-03-02 10:44:45.416092905 isis/src/base/apps/edrget/tsts/connectErrors/truth/ftpFileDNE.txt
<        96 2017-03-02 10:44:45.387092102 isis/src/base/apps/edrget/tsts/connectErrors/truth/httpURLEmpty.txt
<       156 2017-03-02 10:44:45.358091300 isis/src/base/apps/edrget/tsts/connectErrors/truth/httpFileNameEmpty.txt
<       110 2017-03-02 10:44:45.328090470 isis/src/base/apps/edrget/tsts/connectErrors/truth/httpURLInvalid.txt
<        95 2017-03-02 10:44:45.300089695 isis/src/base/apps/edrget/tsts/connectErrors/truth/ftpURLEmpty.txt
<       155 2017-03-02 10:44:45.270088865 isis/src/base/apps/edrget/tsts/connectErrors/truth/ftpFileNameEmpty.txt
<       109 2017-03-02 10:44:45.241088063 isis/src/base/apps/edrget/tsts/connectErrors/truth/ftpURLInvalid.txt
3277,3288d3266
<       224 2014-03-28 16:36:50.000000000 isis/src/mro/apps/hijitter/tsts/default/input/red6.stats.pvl.DIFF~
<       238 2014-03-28 16:36:50.000000000 isis/src/mro/apps/hijitter/tsts/default/input/red6.stats.pvl.DIFF
<       241 2014-03-28 16:36:50.000000000 isis/src/mro/apps/hijitter/tsts/default/input/red5.stats.pvl.DIFF~
<       180 2014-03-28 16:36:50.000000000 isis/src/mro/apps/hijitter/tsts/default/input/red4.stats.pvl.DIFF~
<       176 2014-03-28 16:36:50.000000000 isis/src/mro/apps/hijitter/tsts/default/input/red4.stats.pvl.DIFF
<       311 2014-03-28 16:36:50.000000000 isis/src/mro/apps/hijitter/tsts/default/input/hijitreg.p0745.s1570.def
<        81 2014-03-28 16:36:49.000000000 isis/src/mro/apps/hijitter/tsts/default/input/cktable.csv.DIFF
<  16245124 2014-03-28 16:36:49.000000000 isis/src/mro/apps/hijitter/tsts/default/input/PSP_007556_2010_RED6.balance.cropped.cub
<  16597410 2014-03-28 16:36:49.000000000 isis/src/mro/apps/hijitter/tsts/default/input/PSP_007556_2010_RED5.balance.cropped.cub
<  16256391 2014-03-28 16:36:48.000000000 isis/src/mro/apps/hijitter/tsts/default/input/PSP_007556_2010_RED4.balance.cropped.cub
<     70037 2014-03-28 16:36:48.000000000 isis/src/mro/apps/hijitter/tsts/default/input/PSP_007556_2010.txt
<      2227 2014-03-28 16:36:48.000000000 isis/src/mro/apps/hijitter/tsts/default/input/PSP_007556_2010.cropped.txt
4594d4571
<    197052 2009-07-14 08:24:01.000000000 isis/src/base/apps/edrget/tsts/http/truth/vis1flat.cub
4946d4922
<    477758 2007-08-07 11:35:20.000000000 isis/src/base/apps/edrget/tsts/ftp/truth/f253s05.imq
KrisBecker commented 2 years ago

And here is the result of my run of ctest:


99% tests passed, 12 tests failed out of 2351

Label Time Summary:
apollo          =   4.16 sec*proc (4 tests)
app             = 4508.20 sec*proc (792 tests)
cassini         =   3.70 sec*proc (5 tests)
chandrayaan1    =   0.41 sec*proc (1 test)
clementine      =   0.85 sec*proc (4 tests)
dawn            =   0.78 sec*proc (2 tests)
galileo         =   0.22 sec*proc (1 test)
hayabusa        =   0.90 sec*proc (2 tests)
hayabusa2       =   0.42 sec*proc (1 test)
isis            = 142.25 sec*proc (322 tests)
juno            =   0.33 sec*proc (1 test)
kaguya          =   0.98 sec*proc (2 tests)
lo              =   0.88 sec*proc (3 tests)
lro             =   1.28 sec*proc (3 tests)
mariner         =   1.06 sec*proc (1 test)
messenger       =   0.38 sec*proc (1 test)
mex             =   0.64 sec*proc (2 tests)
mgs             =   2.26 sec*proc (4 tests)
mro             =   3.85 sec*proc (6 tests)
near            =   0.32 sec*proc (1 test)
newhorizons     =   1.13 sec*proc (4 tests)
odyssey         =   0.60 sec*proc (2 tests)
rosetta         =   1.29 sec*proc (2 tests)
tgo             =   0.33 sec*proc (1 test)
unit            = 170.07 sec*proc (377 tests)
viking          =   0.59 sec*proc (1 test)
voyager         =   0.46 sec*proc (1 test)

Total Test time (real) = 6183.62 sec

The following tests did not run:
    1701 - TempTestingFiles.FunctionalTestJitterfitDefault (Disabled)
    1719 - TempTestingFiles.UnitTestImageImporterTestJpeg (Disabled)

The following tests FAILED:
     44 - isis_unit_test_Cube (Failed)
    109 - isis_unit_test_JP2Importer (Failed)
    541 - isis2std_app_test_jpeg2000 (Failed)
    1147 - tgo_module_test_coloredMosaic (Failed)
    1149 - tgo_module_test_mapProjectedReingested (Failed)
    1150 - tgo_module_test_projSingleStichedFrame (Failed)
    1151 - tgo_module_test_singleColorMosaicReingest (Failed)
    1152 - tgo_module_test_uncontrolledSingleColorMosaic (Failed)
    1274 - ThreeImageNetwork.FunctionalTestFindfeaturesErrorNoInput (Failed)
    1715 - TempTestingFiles.FunctionalTestStd2isisJp2 (Failed)
    1721 - TempTestingFiles.UnitTestImageImporterStd2IsisJp2 (Failed)
    1818 - MroHiriseCube.FunctionalTestHimosDefault (Failed)
Errors while running CTest
jessemapel commented 2 years ago

I can confirm that the cron job to sync the test data to s3 each morning is running correctly now. @KrisBecker The errors you are still seeing are already documented in #4666 and we're working on getting them fixed. Once the test data gets updated, it should automatically become available the next day.