Fails in Unit Tests

[ggranato92]

Hi Dominic,

I am working on windows and I encountered some failures when running the unit tests. I am attaching the log file below where you can see the errors. Most of them are related to tolerances but sometimes the error is not specified (it says the application has requested runtime to terminate it in an unusual way).

LogFileTests.txt

[DominicDirkx]

So, just for reference, the failing tests are:

test_TabulatedAerodynamicCoefficients

Here, an error is produced by NRLMSISE, apparently NaN densities are being used

test_OneWayDopplerModel

Here, the problem seems to be tolerances, but they seem to be off by close to 4 orders of magnitude..

test_EstimationFromPositionDoubleDouble

These seem to be very minor tolerance issues, which are solved easily

test_AccelerationPartials

Spice kernel issue

test_MutualSphericalHarmonicPartials

Spice kernel issue

test_PositionPartials

Spice kernel issue

test_PropagationTerminationReason

No idea what's going on here.... Can you run this executable in debug mode and post a stack trace?

test_PointingAnglesCalculator

Small tolerance issue

test_SpiceInterface

No idea what's going on here.... Can you run this executable in debug mode and post a stack trace?

[DominicDirkx]

I've made some quick corrections to:

unitTestPointingAnglesCalculator.cpp unitTestAccelerationPartials.cpp unitTestPositionPartials.cpp unitTestEstimationFromIdealDataDoubleDouble.cpp

See:

https://github.com/Tudat/tudat/commit/992cd783ad2e0843c06b509321ff7938671b0de8

Could you pull the code (development branch) and see if they do indeed fix your problem?

By the way, which compiler are you using?

[DominicDirkx]

I'm not able to reproduce the test_TabulatedAerodynamicCoefficients error in any way. Could you update the file:

tudatBundle/nrlmsise-00/nrlmsise-00.c

by adding

printf("dnet log %e %e %e\n",dm,dd,xm);

on line 203 (somewhere at the beginning of the dnet function, before the if statement), run the test_TabulatedAerodynamicCoefficients executable, and post the output here? Thanks

[ggranato92]

I am unfortunately having troubles performing the debugging of test_PropagationTerminationReason because of some errors during the building phase (in debug mode). Attached is a screenshot with the strange issues that occur.

Concerning the test_SpiceInterface I manage to do the debugging but it does not show any error...

The TabulatedAerodyamicCoefficients test output (after the update yo suggested) is:

This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information. Running 1 test case... dnet log 1.#QNAN0e+000 1.#QNAN0e+000 2.800000e+001 dnet log error 1.#QNAN0e+000 1.#QNAN0e+000 2.800000e+001 dnet log 1.#QNAN0e+000 1.#QNAN0e+000 4.000000e+000 dnet log error 1.#QNAN0e+000 1.#QNAN0e+000 4.000000e+000 dnet log 1.#QNAN0e+000 1.#QNAN0e+000 1.600000e+001 dnet log error 1.#QNAN0e+000 1.#QNAN0e+000 1.600000e+001 dnet log 1.#QNAN0e+000 1.#QNAN0e+000 3.200000e+001 dnet log error 1.#QNAN0e+000 1.#QNAN0e+000 3.200000e+001 dnet log 1.#QNAN0e+000 1.#QNAN0e+000 4.000000e+001 dnet log error 1.#QNAN0e+000 1.#QNAN0e+000 4.000000e+001 dnet log 1.#QNAN0e+000 1.#QNAN0e+000 1.000000e+000 dnet log error 1.#QNAN0e+000 1.#QNAN0e+000 1.000000e+000 dnet log 1.#QNAN0e+000 1.#QNAN0e+000 1.400000e+001 dnet log error 1.#QNAN0e+000 1.#QNAN0e+000 1.400000e+001 C:\tudatBundle\tudat\bin\unit_tests\test_TabulatedAerodynamicCoefficients.exe exited with code 3

Finally, after merging with the development branch in order to update everything, the tests that fail are the following:

7 - test_TabulatedAerodynamicCoefficients (Failed) 11 - test_WindModel (Not Run) 31 - test_EmpiricalAccelerations (Not Run) 43 - test_TabulatedRotationalEphemeris (Not Run) 72 - test_OneWayDopplerModel (Failed) 77 - test_EstimationFromPositionDoubleDouble (Failed) 83 - test_ParameterInfluenceDetermination (Not Run) 84 - test_AccelerationPartials (Failed) 86 - test_MutualSphericalHarmonicPartials (Failed) 90 - test_OneWayDopplerPartials (Failed) 92 - test_PositionPartials (Failed) 109 - test_PropagationTerminationReason (Failed) 110 - test_RotationalDynamicsPropagation (Not Run) 111 - test_CR3BPPropagation (Not Run) 117 - test_PointingAnglesCalculator (Failed) Errors while running CTest Makefile:71: recipe for target 'test' failed mingw32-make.exe: *** [test] Error 8 20:06:37: The process "C:\Program Files (x86)\CMake\bin\cmake.exe" exited with code 2. Error while building/deploying project TudatBundle (kit: Desktop Qt 5.9.1 MinGW 32bit) When executing step "CMake Build"

I am wondering why some of them are not run.

The compiler that I am using is MingW (I am working on windows)

[DominicDirkx]

Hi Giovanni,

So, this has made things worse... Is this in Debug or Release mode?

Could you check in your code whether the corrections from the last couple of commits have been correctly merged?

Have you recompiled all tests before running them?

Could you attach the new log file?

Thanks!

Dominic

[DominicDirkx]

Based on discussion in #227, I've made some changes to how multiple orbit kernels are loaded. Could you pull the latest code and recompile/run test_MutualSphericalHarmonicPartials and test_ParameterInfluenceDetermination ?

[ggranato92]

Hi Dominic,

I have used debug mode only for those couple of tests indicated by you. Concerning the last list of tests failed that I have reported, they were not run in debug mode.

I have merged with the latest version of the development branch and have checked that the commits were actually applied. I remember the first time I merged there were some conflicts but then, by doing first git stash and then merging, everything went fine.

Right now, to make sure I didn't make mistakes I am trying to compile everything again with the latest updates but I am continuously getting the memory error... (my laptop is 8 GB RAM and I have also removed the -j2). What can I do?

Thank you, Giovanni

[DominicDirkx]

Hi Giovanni,

Thanks for the quick reply. What do you mean by 'the memory error'? If you mean that you are running out of RAM, could you let me know in the compilation of which executable this is occuring? You can attach your compilation output if you're not sure,

Dominic

[ggranato92]

The error is "Out of memory allocating 214620 bytes" which I remember is the same error I used to get with my old laptop.

Attached is the compile output Compile output.txt

[DominicDirkx]

For now, toggle the COMPILE_HIGH_ACCURACY_ESTIMATION_TESTS cmake argument in the Project tab to OFF (see attachment).

This should prevent this (and other) tests from building. However, before doing so, could you check if your computer is really out of RAM when compiling?

[ggranato92]

This is really strange. I have actually checked and I do not run out of RAM while compiling...But at a certain point that error occurs. I am noticing that when I click on the issue I get "File not found: cc1plus.exe". I am going to follow your advice and will let you know

[ggranato92]

I have turned off the COMPILE_HIGH_ACCURACY_ESTIMATION_TESTS but I am getting exactly the same error

[DominicDirkx]

Have you clicked the 'Apply Configuration Changes' button? Does it occur on the compilation of the same executable?

[ggranato92]

You are right, I hadn't. Now it works and no issues (only the usual warnings)

[DominicDirkx]

Excellent! What does running the unit tests yield as output?

[ggranato92]

I am attaching the log files here. Unfortunately there are still failures Last_tests.txt

Tests_failed.txt

[DominicDirkx]

That's very strange. Some of these I had expected to fail, but there are a bunch of new failures now...

Do you know which revision were you on before pulling the latest code?

Could you get the differences of your branch w.r.t. the development branch:

git diff upstream/development

(change the name of the remote accordingly if it is not called upstream). You can go ahead and remove the added line in NRLMSISE, it seems that, for that particular test, the input of densities is always NaN.

[ggranato92]

Hi Dominic,

After recompiling everything and running the unit tests I get the following results (see attachments).

Last_tests.txt

Last_tests_failed.txt

Giovanni

[DominicDirkx]

Hi Giovanni,

On my home Windows computer, I was able to reproduce the failures in:

test_OneWayDopplerModel test_OneWayDopplerPartials test_AccelerationPartials

I have made corrections to the code so that these no longer occur. The problem was:

test_OneWayDopplerModel: unrealistically small tolerances test_OneWayDopplerPartials: non-optimal time step in numerical difference used in test test_AccelerationPartials: Spice kernel load issue

Other issues I was not able to reproduce, but we're slowly but surely converging on a solution to these windows issues...

[ggranato92]

Hi Dominic,

I have had some time to perform the unit tests again, after updating the tudat. I am attaching here the results. I notice that some tests are not performed at all because the executable is not found... (by the way, I turned the compile_high_accuracy_estimation_test on again and gave no issues)

Last_test.txt Last_test_fail.txt

Best, Giovanni

[DominicDirkx]

Hi Giovanni,

it looks like it is running an old version of the code (the naif0009.tls file was replaced with teh newer naif0012.tls). Are you sure that you:

• Clicked 'Apply Configuration Changes'
• Recompiled the whole code (without the test compile argument)
• Have not switched between debug/default/release mode

Let me know,

Cheers,

Dominic

[ggranato92]

Hi Dominic,

I've made sure to recompile everything from the start and applied all the configuration changes before running the tests again. Here is the output:

Last_test.txt Last_test_fail.txt

Giovanni

[DominicDirkx]

Your code tries to load the naif0009.tls file, which it shouldn't.... Could you check in the code for one of the failed unit tests whether this is also in your code? (Spice kernels may be loaded by the loadStandardSpiceKernels function).

[ggranato92]

The strange thing is that in my code that line is correct (naif0012.tls and no more naif0009.tls). Anyway I have performed the tests again and I notice that the LastTest and LastTestsFailed files are not updated anymore.. I am attaching here the compile output of the unit tests

Compile_output_test.txt

Do you know the reason why the log files are no longer created?

[DominicDirkx]

Hi Giovanni,

It seems that your tests are terminating at test 78, is this correct? Are there RAM (or other) issues when doing this test?

Dominic

[ggranato92]

I have just checked through the task manager if there are memory issues but apparently there are no problems with the RAM. When the process stops there is still a lot of memory available..

[ggranato92]

And concerning the point where tests are terminating: when I use -j2 they are terminating at test 78, when I remove -j2 they terminate at test 77

[DominicDirkx]

In the Astrodynamics/OrbitDetermination/UnitTests/orbitDeterminationTestCases file, change:

boost::shared_ptr< TranslationalStatePropagatorSettings< StateScalarType > > propagatorSettings =
        boost::make_shared< TranslationalStatePropagatorSettings< StateScalarType > >
        ( centralBodies, accelerationModelMap, bodiesToIntegrate,
          getInitialStateVectorOfBodiesToEstimate( parametersToEstimate ),
          TimeType( finalEphemerisTime + 4.0 * maximumTimeStep ),
          cowell, boost::shared_ptr< DependentVariableSaveSettings >( ), 600.0 );

to:

boost::shared_ptr< TranslationalStatePropagatorSettings< StateScalarType > > propagatorSettings =
        boost::make_shared< TranslationalStatePropagatorSettings< StateScalarType > >
        ( centralBodies, accelerationModelMap, bodiesToIntegrate,
          getInitialStateVectorOfBodiesToEstimate( parametersToEstimate ),
          TimeType( finalEphemerisTime + 4.0 * maximumTimeStep ),
          cowell, boost::shared_ptr< DependentVariableSaveSettings >( ) );

removing the '600.0' at the end. Recompile just a single test (the double-double estimation for instance) and see if runs correctly.

[ggranato92]

I have made that modification in the code and the test_EstimationFromPositionDoubleDouble works correctly. No errors detected after recompiling the single test and running it.

[DominicDirkx]

Ok, good. What if you now compile all the tests again, and then run them?

[ggranato92]

Now everything works properly and the log files are created. There are still failures but much less than before. The following tests have failed:

7:test_TabulatedAerodynamicCoefficients 72:test_DopplerModels 80:test_SphericalHarmonicPartials 105:test_PropagationTerminationReason

Here I am attaching the log file Last_test.txt

[DominicDirkx]

Most of these have been resolved in development-json.

The remaining issue is: #273