search

OpenMDAO / Aviary

NASA's aircraft analysis, design, and optimization tool
https://openmdao.github.io/Aviary/
Other
107 stars 51 forks source link

Three `bench_test` failures #322

Open swryan opened 3 weeks ago

swryan commented 3 weeks ago

Description

There are 3 bench_test failures when running on the cluster:

The following tests failed:
test_FLOPS_based_sizing_N3CC.py:ProblemPhaseTestCase.bench_test_sizing_N3CC
test_NPSS_builder.py:AviaryNPSSTestCase.bench_test_aviary_NPSS
test_cmd_entry_points.py:CommandEntryPointsTestCases.bench_test_SNOPT_cmd

Passed:  11
Failed:  3
Skipped: 0

Example

testflo -vs aviary/interface/test/test_cmd_entry_points.py:CommandEntryPointsTestCases.bench_test_SNOPT_cmd --testmatch=bench_test*

 aviary run_mission aviary/models/test_aircraft/aircraft_for_bench_GwGm.csv --optimizer SNOPT --max_iter 1
Traceback (most recent call last):
  File "/home/swryan/miniconda3/envs/aviary/bin/aviary", line 8, in <module>
    sys.exit(aviary_cmd())
             ^^^^^^^^^^^^
  File "/home/swryan/dev/Aviary/aviary/interface/cmd_entry_points.py", line 122, in aviary_cmd
    options.executor(options, user_args)
  File "/home/swryan/dev/Aviary/aviary/interface/methods_for_level1.py", line 203, in _exec_level1
    prob = run_level_1(
           ^^^^^^^^^^^^
  File "/home/swryan/dev/Aviary/aviary/interface/methods_for_level1.py", line 135, in run_level_1
    prob = run_aviary(input_deck, phase_info, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/swryan/dev/Aviary/aviary/interface/methods_for_level1.py", line 70, in run_aviary
    prob.load_inputs(aircraft_filename, phase_info, verbosity=verbosity)
  File "/home/swryan/dev/Aviary/aviary/interface/methods_for_level2.py", line 306, in load_inputs
    phase_info, _ = phase_info_parameterization(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnboundLocalError: cannot access local variable 'phase_info_parameterization' where it is not associated with a value

testflo -vs aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py --testmatch=bench_test*

aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py:ProblemPhaseTestCase.bench_test_sizing_N3CC  ... FAIL (00:00:3.72, 284 MB)

[ ... a bunch of warnings deleted ... ]

Traceback (most recent call last):
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/openmdao/solvers/linear/direct.py", line 320, in _linearize
    self._lu = scipy.sparse.linalg.splu(matrix)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/scipy/sparse/linalg/_dsolve/linsolve.py", line 437, in splu
    return _superlu.gstrf(N, A.nnz, A.data, indices, indptr,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Factor is exactly singular

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/swryan/dev/Aviary/aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py", line 445, in bench_test_sizing_N3CC
    prob = run_trajectory(sim=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/swryan/dev/Aviary/aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py", line 427, in run_trajectory
    dm.run_problem(prob, simulate=sim, make_plots=False, simulate_kwargs={
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/dymos/run_problem.py", line 96, in run_problem
    failed = _refine_iter(problem, refine_iteration_limit, refine_method, case_prefix=case_prefix,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/dymos/grid_refinement/refinement.py", line 40, in _refine_iter
    failed = problem.run_driver(case_prefix=case_prefix if refine_iteration_limit > 0 else _case_prefix,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/openmdao/core/problem.py", line 737, in run_driver
    return driver._run()
           ^^^^^^^^^^^^^
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/openmdao/core/driver.py", line 751, in _run
    self.result.success = not self.run()
                              ^^^^^^^^^^
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/openmdao/drivers/pyoptsparse_driver.py", line 623, in run
    raise exc_info[1].with_traceback(exc_info[2])
  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/openmdao/drivers/pyoptsparse_driver.py", line 735, in _objfunc
    self._run_solve_nonlinear()

[ ... a bunch of stuff deleted ... ]

  File "/home/swryan/miniconda3/envs/aviary/lib/python3.12/site-packages/openmdao/solvers/linear/direct.py", line 322, in _linearize
    raise RuntimeError(format_singular_error(system, matrix))
RuntimeError: NaN entries found in 'traj.phases.climb.rhs_all.core_aerodynamics.SkinFrictionCoef' <class SkinFriction> for rows associated with states/residuals ['cf_iter', 'wall_temp'].

testflo -vs aviary/examples/external_subsystems/engine_NPSS/test_NPSS_builder.py --testmatch=benchtest* (requires NPSS in the environment, with NPSSTOP, etc. environment variables)

/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/examples/external_subsystems/engine_NPSS/test_NPSS_builder.py:AviaryNPSSTestCase.bench_test_aviary_NPSS  ... FAIL (00:00:23.77, 372 MB)

[ ... a bunch of warnings deleted ... ]

/mdao/u/swryan/.conda/envs/Aviary_20240612-000004/lib/python3.12/site-packages/openmdao/recorders/sqlite_recorder.py:226: UserWarning:The existing case recorder file, aviary_history.db, is being overwritten.

Traceback (most recent call last):
  File "/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/examples/external_subsystems/engine_NPSS/test_NPSS_builder.py", line 23, in bench_test_aviary_NPSS
    assert_near_equal(prob.get_val('aircraft:engine:design_mass_flow'),
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mdao/u/swryan/.conda/envs/Aviary_20240612-000004/lib/python3.12/site-packages/openmdao/core/problem.py", line 545, in get_val
    val = self.model.get_val(name, units=units, indices=indices, get_remote=get_remote,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mdao/u/swryan/.conda/envs/Aviary_20240612-000004/lib/python3.12/site-packages/openmdao/core/system.py", line 5267, in get_val
    raise KeyError('{}: Variable "{}" not found.'.format(self.msginfo, name))
KeyError: '<model> <class AviaryGroup>: Variable "aircraft:engine:design_mass_flow" not found.'

Aviary Version

0.9.4-dev

Relevant environment information

This is coming from the nightly run on the cluster using the latest development versions of OpenMDAO, pyCycle and Dymos.

Kenneth-T-Moore commented 3 weeks ago

bench_test_sizing_N3CC is fixed on #314 bench_test_aviary_NPSS should be fixed on Jason's next PR. bench_test_SNOPT_cmd is a new failure.

swryan commented 3 weeks ago

bench_test_SNOPT_cmd was fixed by #321

swryan commented 2 weeks ago

314 was merged, so the only remaining failure is:

Traceback (most recent call last):
  File "/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/examples/external_subsystems/engine_NPSS/test_NPSS_builder.py", line 23, in bench_test_aviary_NPSS
    assert_near_equal(prob.get_val('aircraft:engine:design_mass_flow'),
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mdao/u/swryan/.conda/envs/Aviary_20240618-000005/lib/python3.12/site-packages/openmdao/core/problem.py", line 545, in get_val
    val = self.model.get_val(name, units=units, indices=indices, get_remote=get_remote,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mdao/u/swryan/.conda/envs/Aviary_20240618-000005/lib/python3.12/site-packages/openmdao/core/system.py", line 5256, in get_val
    raise KeyError('{}: Variable "{}" not found.'.format(self.msginfo, name))
KeyError: '<model> <class AviaryGroup>: Variable "aircraft:engine:design_mass_flow" not found.'

The following tests failed:
test_NPSS_builder.py:AviaryNPSSTestCase.bench_test_aviary_NPSS

Passed:  13
Failed:  1
Skipped: 0
swryan commented 2 weeks ago

After #331 was merged yesterday, there are two different failures:

/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/examples/external_subsystems/engine_NPSS/test_NPSS_builder.py:AviaryNPSSTestCase.bench_test_aviary_NPSS  ... FAIL (00:00:0.33, 219 MB)
Traceback (most recent call last):
  File "/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/examples/external_subsystems/engine_NPSS/test_NPSS_builder.py", line 17, in bench_test_aviary_NPSS
    prob = define_aviary_NPSS_problem()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/examples/external_subsystems/engine_NPSS/define_simple_engine_problem.py", line 23, in define_aviary_NPSS_problem
    prob.load_inputs('models/test_aircraft/aircraft_for_bench_FwFm.csv',
TypeError: AviaryProblem.load_inputs() got an unexpected keyword argument 'engine_builder'

and

/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py:ProblemPhaseTestCase.bench_test_sizing_N3CC  ... FAIL (00:00:1.20, 242 MB)

[ ... warnings deleted ... ]

Traceback (most recent call last):
  File "/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py", line 460, in bench_test_sizing_N3CC
    prob = run_trajectory(sim=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mdao/u/swryan/benchmark/runner/repos/Aviary/Aviary/aviary/validation_cases/benchmark_tests/test_FLOPS_based_sizing_N3CC.py", line 394, in run_trajectory
    prob.setup(force_alloc_complex=True)
  File "/mdao/u/swryan/.conda/envs/Aviary_20240619-000004/lib/python3.12/site-packages/openmdao/core/problem.py", line 1035, in setup
    self._check_collected_errors()
  File "/mdao/u/swryan/.conda/envs/Aviary_20240619-000004/lib/python3.12/site-packages/openmdao/core/problem.py", line 633, in _check_collected_errors
    raise exc_type('\n'.join(final_msg)).with_traceback(tback)
  File "/mdao/u/swryan/.conda/envs/Aviary_20240619-000004/lib/python3.12/site-packages/openmdao/core/group.py", line 2126, in _setup_global_connections
    self._collect_error(msg)
RuntimeError: 
Collected errors for problem 'problem':
   <model> <class Group>: Attempted to connect from 'mission:design:fuel_mass' to 'regularization.fuel_mass', but 'mission:design:fuel_mass' is an input. All connections must be from an output to an input.