Long CI times

[eivindjahren]

CI times have nearly doubled since august. Here are the list of top times taken. We should attempt to have them run much faster:

=================================================== fixture duration top ===================================================
total          name                                                                      num   avg            min           
0:05:57.480591                                                    _shared_snake_oil_case    34 0:00:00.000408 0:00:00.000259
0:00:42.789007                                                           template_config     2 0:00:21.394504 0:00:14.018795
0:00:36.595306                                               poly_example_tmp_dir_shared     2 0:00:18.297653 0:00:17.915451
0:00:23.176782                                                    snake_oil_case_storage    31 0:00:00.739975 0:00:00.460819
0:00:17.767035                                                         batch_sim_example    25 0:00:00.676780 0:00:00.534848
0:00:09.869191                                                            snake_oil_case    14 0:00:00.722335 0:00:00.196966
0:00:08.441525                                                                ert_config    13 0:00:00.629636 0:00:00.415449
0:00:04.493270                                                       dark_storage_client    13 0:00:00.325717 0:00:00.221057
0:00:03.872375                                                            set_ert_config    14 0:00:00.001219 0:00:00.000144
0:00:03.775739                                                             writing_setup     6 0:00:00.594805 0:00:00.530015
0:00:03.487874                                                   snake_oil_field_example     3 0:00:01.190046 0:00:00.639722
0:00:03.227112                                                                 poly_case    11 0:00:00.153200 0:00:00.141793
0:00:02.603571                                                                 evaluator     4 0:00:00.475682 0:00:00.270320
0:00:02.429399                                               copy_snake_oil_case_storage    34 0:00:00.072030 0:00:00.021234
0:00:02.051097                                                              minimum_case     4 0:00:00.490262 0:00:00.290526
0:00:01.363080                                              snake_oil_export_runpath_job     2 0:00:00.681540 0:00:00.679090
0:00:01.213028                                                                  tmp_path  1074 0:00:00.000796 0:00:00.000209
0:00:00.773999                                                                      data     5 0:00:00.156734 0:00:00.093675
0:00:00.626981                                                                  env_save  1817 0:00:00.000277 0:00:00.000092
0:00:00.605276                                                             poly_template     1 0:00:00.605276 0:00:00.605276
0:00:00.502330                                                                 log_check  1817 0:00:00.000197 0:00:00.000060
0:00:00.244148                                                           no_cert_in_test  1712 0:00:00.000110 0:00:00.000043
0:00:00.231983                                                                    server    10 0:00:00.024773 0:00:00.002437
0:00:00.195749                                                                   storage   202 0:00:00.001067 0:00:00.000246
0:00:00.145122                                                            prior_ensemble    37 0:00:00.001104 0:00:00.000788
0:00:00.141772                                                               monkeypatch  1789 0:00:00.000059 0:00:00.000020
0:00:00.107226                                                            copy_poly_case    39 0:00:00.002282 0:00:00.001085
0:00:00.059286                                                                use_tmpdir   443 0:00:00.000085 0:00:00.000028
0:00:00.050067                                                          setup_mock_queue    12 0:00:00.004119 0:00:00.002034
0:00:00.040536                                                         snake_oil_storage     8 0:00:00.004593 0:00:00.003694
0:08:49.010827                                                               grand total 12956 0:00:00.000086 0:00:00.000019
================================================== test call duration top ==================================================
total          name                                                                      num   avg             min           
0:04:20.115774                                                     test_memory_smoothing     1 0:04:20.115774 0:04:20.115774
0:02:48.137471                             test_that_stop_on_fail_workflow_jobs_stop_ert    16 0:00:10.519090 0:00:09.151295
0:02:32.146247             test_that_adaptive_localization_with_cutoff_0_equals_ESupdate     2 0:01:16.073124 0:00:50.457421
0:02:13.581242              test_that_posterior_generalized_variance_increases_in_cutoff     2 0:01:06.790621 0:00:39.855939
0:02:06.739168       test_that_adaptive_localization_with_cutoff_1_equals_ensemble_prior     2 0:01:03.369584 0:00:31.176598
0:01:46.185789                                                             test_tracking     8 0:00:12.362905 0:00:09.170453
0:01:40.221771                                                                  test_ies     2 0:00:50.110885 0:00:35.898085
0:01:34.767010                                                test_update_multiple_param     2 0:00:47.383505 0:00:28.329777
0:01:20.997474                     test_surface_init_fails_during_forward_model_callback     2 0:00:40.498737 0:00:15.346728
0:01:18.086057                 test_that_prior_is_not_overwritten_in_ensemble_experiment     6 0:00:12.170947 0:00:11.518403
0:01:09.257066                         test_that_posterior_has_lower_variance_than_prior     2 0:00:34.628533 0:00:22.532657
0:01:07.607160                                        test_batch_ctx_status_failing_jobs     2 0:00:33.803580 0:00:21.145697
0:01:02.437524                    test_parameter_update_with_inactive_cells_xtgeo_grdecl     1 0:01:02.437524 0:01:02.437524
0:01:00.381541                                                               test_es_mda     2 0:00:30.190771 0:00:30.107611
0:00:52.882296                                                   test_field_param_update     1 0:00:52.882296 0:00:52.882296
0:00:51.150592                                                 test_run_mocked_lsf_queue     2 0:00:25.575296 0:00:21.640974
0:00:45.890613                                       test_setting_env_context_during_run     4 0:00:09.172860 0:00:09.165491
0:00:44.555640 test_that_running_ies_with_different_steplength_produces_different_result     1 0:00:44.555640 0:00:44.555640
0:00:43.915579        test_that_surfaces_retain_their_order_when_loaded_and_saved_by_ert     2 0:00:21.957790 0:00:21.825076
0:00:36.841740                                                         test_runpath_file     7 0:00:00.007112 0:00:00.001116
0:00:35.561905                                                   test_ensemble_evaluator     2 0:00:17.780953 0:00:17.675266
0:00:34.873145                                test_ensemble_evaluator_disable_monitoring     2 0:00:17.436572 0:00:17.407545
0:00:29.683545                                            test_shell_scripts_integration     2 0:00:14.841773 0:00:13.802011
0:00:29.339217  test_that_the_model_warns_when_active_realizations_less_min_realizations     2 0:00:14.669608 0:00:13.867328
0:00:27.559337            test_that_multiple_keyword_specific_tokens_are_located_shuffle     1 0:00:27.559337 0:00:27.559337
0:00:23.663846                                                   test_field_param_memory     1 0:00:23.663846 0:00:23.663846
0:00:22.745755                                                 test_tracking_missing_ecl     2 0:00:11.372878 0:00:11.362785
0:00:22.669344                       test_that_setenv_sets_environment_variables_in_jobs     2 0:00:11.334672 0:00:09.314593
0:00:21.555529                                        test_failing_job_cli_error_message     2 0:00:10.777765 0:00:09.760073
0:00:20.045380                                                 test_surface_param_update     1 0:00:20.045380 0:00:20.045380
0:44:45.908149                                                               grand total  1815 0:00:00.034141 0:00:00.000190
================================================= test setup duration top ==================================================
total          name                                                                      num   avg            min           
0:01:17.071907                                                        test_export_misfit     1 0:01:17.071907 0:01:17.071907
0:01:16.381220                                                  test_keyword_type_checks     1 0:01:16.381220 0:01:16.381220
0:01:15.736252                                                        test_update_report     2 0:00:37.868126 0:00:00.805690
0:01:15.007038                                                    test_summary_collector     1 0:01:15.007038 0:01:15.007038
0:00:39.832978                                    test_that_different_length_values_fail     1 0:00:39.832978 0:00:39.832978
0:00:18.186308                                                          test_history_obs     1 0:00:18.186308 0:00:18.186308
0:00:08.450207               test_that_batch_simulation_handles_invalid_suffixes_at_init    12 0:00:00.727408 0:00:00.540720
0:00:05.921568                                                    test_run_path_stripped     7 0:00:00.846073 0:00:00.669619
0:00:03.757847                      test_that_starting_with_invalid_key_raises_key_error     5 0:00:00.770114 0:00:00.649042
0:00:03.021712               test_that_batch_simulator_handles_invalid_suffixes_at_start     4 0:00:00.675782 0:00:00.629409
0:00:02.675885                                                      test_update_snapshot     3 0:00:00.849517 0:00:00.838019
0:00:01.936193                                          test_write_snakeoil_runpath_file     4 0:00:00.467772 0:00:00.442079
0:00:01.661642                                                           test_all_active     1 0:00:01.661642 0:00:01.661642
0:00:01.578704                                                     test_batch_simulation     2 0:00:00.789352 0:00:00.649606
0:00:01.383836                                                        test_dying_batcher     1 0:00:01.383836 0:00:01.383836
0:00:01.378773                                                         test_localization     2 0:00:00.689387 0:00:00.658964
0:00:01.193010           test_write_to_runpath_produces_the_transformed_field_in_storage     1 0:00:01.193010 0:00:01.193010
0:00:01.168825                                            test_batch_simulation_suffixes     2 0:00:00.584413 0:00:00.576504
0:00:01.167898                                                        test_gen_kw_priors     1 0:00:01.167898 0:00:01.167898
0:00:01.081353                                                   test_all_data_type_keys     2 0:00:00.540676 0:00:00.004709
0:00:01.053020                                test_update_only_using_subset_observations     1 0:00:01.053020 0:00:01.053020
0:00:01.043073                                         test_observation_keys_missing_key     1 0:00:01.043073 0:00:01.043073
0:00:01.004645                               test_summary_data_verify_indices_and_values     1 0:00:01.004645 0:00:01.004645
0:00:00.990758                                                   test_gen_data_collector     1 0:00:00.990758 0:00:00.990758
0:00:00.947224                                                   test_get_coeffs_records     3 0:00:00.314170 0:00:00.305438
0:00:00.928295                                                   test_load_forward_model     2 0:00:00.464147 0:00:00.001516
0:00:00.886996                                                              test_gen_obs     1 0:00:00.886996 0:00:00.886996
0:00:00.805369                                                  test_gen_obs_and_summary     1 0:00:00.805369 0:00:00.805369
0:00:00.797335                                                         test_summary_keys     1 0:00:00.797335 0:00:00.797335
0:00:00.790150                                                     test_gen_kw_collector     1 0:00:00.790150 0:00:00.790150
0:07:24.477417                                                               grand total  1880 0:00:00.002920 0:00:00.000168
================================================ test teardown duration top ================================================
total          name                                                                      num   avg            min           
0:00:01.656258                            test_new_monitor_can_pick_up_where_we_left_off     1 0:00:01.656258 0:00:01.656258
0:00:00.568089                                                   test_get_coeffs_records     3 0:00:00.182125 0:00:00.149763
0:00:00.240397                                          test_get_experiment_observations     1 0:00:00.240397 0:00:00.240397
0:00:00.238190                                              test_get_ensemble_parameters     1 0:00:00.238190 0:00:00.238190
0:00:00.222822                                                         test_get_response     1 0:00:00.222822 0:00:00.222822
0:00:00.217982                                                         test_get_ensemble     1 0:00:00.217982 0:00:00.217982
0:00:00.212165                                                       test_get_experiment     1 0:00:00.212165 0:00:00.212165
0:00:00.206277                                              test_get_record_observations     1 0:00:00.206277 0:00:00.206277
0:00:00.201584                                                      test_misfit_endpoint     1 0:00:00.201584 0:00:00.201584
0:00:00.199998                                                       test_refresh_facade     1 0:00:00.199998 0:00:00.199998
0:00:00.198024                                              test_get_experiment_ensemble     1 0:00:00.198024 0:00:00.198024
0:00:00.197954                                      test_get_responses_with_observations     1 0:00:00.197954 0:00:00.197954
0:00:00.133056                                           test_rangestring_to_list_errors    10 0:00:00.003190 0:00:00.001642
0:00:00.086453                                                  test_rangestring_to_mask    24 0:00:00.002016 0:00:00.001804
0:00:00.067225                                                    test_gen_kw_trans_func    47 0:00:00.001101 0:00:00.000939
0:00:00.055898               test_that_batch_simulation_handles_invalid_suffixes_at_init    12 0:00:00.002260 0:00:00.002164
0:00:00.054382                                                  test_mask_to_rangestring    12 0:00:00.002146 0:00:00.001726
0:00:00.053019                                           test_rangestring_to_mask_errors    20 0:00:00.002206 0:00:00.001732
0:00:00.046185                                                   test_custom_log_capture    25 0:00:00.001798 0:00:00.001235
0:00:00.035927                                              test_check_if_runpath_exists    14 0:00:00.002481 0:00:00.002389
0:00:00.034071                      test_that_starting_with_invalid_key_raises_key_error     5 0:00:00.003653 0:00:00.003266
0:00:00.033967                             test_that_stop_on_fail_workflow_jobs_stop_ert    16 0:00:00.001795 0:00:00.000590
0:00:00.030822                                     test_that_sampling_is_fixed_from_name     9 0:00:00.003381 0:00:00.002721
0:00:00.027474                               test_that_killing_killed_job_does_not_raise     1 0:00:00.027474 0:00:00.027474
0:00:00.027240                                              test_direct_dark_performance    48 0:00:00.000312 0:00:00.000157
0:00:00.025536                                                 test_gen_kw_is_log_or_not    10 0:00:00.002503 0:00:00.001386
0:00:00.023322                                                       test_run_torque_job     6 0:00:00.004179 0:00:00.001215
0:00:00.022208                                                       test_simulation_job     3 0:00:00.002357 0:00:00.002280
0:00:00.021983              test_inter_region_summary_format_contains_in_and_out_regions    25 0:00:00.000857 0:00:00.000785
0:00:00.021823                                                test_gen_kw_params_parsing    19 0:00:00.001078 0:00:00.000923
0:00:07.733711                                                               grand total  1880 0:00:00.001209 0:00:00.000157

[eivindjahren]

Update as of fd17a0fd58cebebfbe5c812f2a2f2c58c2295dbe

===================================================== fixture duration top ======================================================
total          name                                                                           num avg            min           
0:07:04.635041                                                         _shared_snake_oil_case    26 0:00:00.000382 0:00:00.000209
0:00:56.124233                                                                template_config    2 0:00:28.062117 0:00:11.715778
0:00:21.764215                                                    poly_example_tmp_dir_shared    1 0:00:21.764215 0:00:21.764215
0:00:16.133149                                                         snake_oil_case_storage    24 0:00:00.582030 0:00:00.291991
0:00:10.348173                                                                      evaluator    4 0:00:02.371425 0:00:01.077176
0:00:09.958596                                                                 snake_oil_case    14 0:00:00.684376 0:00:00.185418
0:00:09.398978                                                              batch_sim_example    25 0:00:00.358005 0:00:00.201860
0:00:07.781310                                                            dark_storage_client    13 0:00:00.425745 0:00:00.318779
0:00:07.371727                                                                     ert_config    15 0:00:00.445702 0:00:00.256470
0:00:06.928130                                                                      poly_case    11 0:00:00.523352 0:00:00.286875
0:00:05.175852                                                                  writing_setup    6 0:00:00.795098 0:00:00.422288
0:00:03.116717                                                        snake_oil_field_example    3 0:00:01.090098 0:00:00.434989
0:00:02.459991                                                    copy_snake_oil_case_storage    26 0:00:00.056148 0:00:00.029638
0:00:01.966869                                                                 set_ert_config    14 0:00:00.000462 0:00:00.000059
0:00:01.944181                                                                   minimum_case    4 0:00:00.474210 0:00:00.292980
0:00:01.733482                                                                       tmp_path  1072 0:00:00.000956 0:00:00.000247
0:00:01.426571                                                   snake_oil_export_runpath_job    2 0:00:00.713286 0:00:00.624175
0:00:00.859032                                                                       env_save  1826 0:00:00.000314 0:00:00.000092
0:00:00.724504                                                                           data    6 0:00:00.079389 0:00:00.057829
0:00:00.632449                                                                      log_check  1826 0:00:00.000210 0:00:00.000091
0:00:00.372440                                                                  poly_template    1 0:00:00.372440 0:00:00.372440
0:00:00.315895                                                                no_cert_in_test  1684 0:00:00.000123 0:00:00.000046
0:00:00.282166                                                                        storage   204 0:00:00.000751 0:00:00.000260
0:00:00.210011                                                                 prior_ensemble    38 0:00:00.001326 0:00:00.000563
0:00:00.184723                                                                    monkeypatch  1796 0:00:00.000068 0:00:00.000020
0:00:00.128645                                                                 copy_poly_case    39 0:00:00.002263 0:00:00.000951
0:00:00.103134                                                               setup_mock_queue    12 0:00:00.003940 0:00:00.002449
0:00:00.081922                                                                     use_tmpdir   446 0:00:00.000096 0:00:00.000032
0:00:00.076767                                                                            api    7 0:00:00.005776 0:00:00.005152
0:00:00.060067                                                                  full_snapshot    3 0:00:00.016806 0:00:00.014279
0:09:53.265881                                                                    grand total 129 59 0:00:00.000081 0:00:00.000019
==================================================== test call duration top =====================================================
total          name                                                                           num avg            min           
0:05:22.729885                                                          test_memory_smoothing    1 0:05:22.729885 0:05:22.729885
0:03:48.223872                                  test_that_stop_on_fail_workflow_jobs_stop_ert    16 0:00:13.101348 0:00:09.600405
0:02:31.590438                                                                  test_tracking    8 0:00:16.560646 0:00:11.780211
0:01:38.394214                  test_that_adaptive_localization_with_cutoff_0_equals_ESupdate    2 0:00:49.197107 0:00:47.631634
0:01:37.377075                   test_that_posterior_generalized_variance_increases_in_cutoff    2 0:00:48.688538 0:00:47.252681
0:01:34.763829                          test_surface_init_fails_during_forward_model_callback    2 0:00:47.381915 0:00:22.083319
0:01:18.625825                                                                       test_ies    2 0:00:39.312913 0:00:37.468610
0:01:14.469168                                     test_ensemble_evaluator_disable_monitoring    2 0:00:37.234584 0:00:29.275344
0:01:07.427462                                                     test_update_multiple_param    2 0:00:33.713731 0:00:32.529626
0:01:04.871940                      test_that_prior_is_not_overwritten_in_ensemble_experiment    6 0:00:10.583726 0:00:09.799250
0:00:57.878928                                                                    test_es_mda    2 0:00:28.939464 0:00:27.910214
0:00:52.068368                                                      test_run_mocked_lsf_queue    2 0:00:26.034184 0:00:22.745264
0:00:50.364827            test_that_adaptive_localization_with_cutoff_1_equals_ensemble_prior    2 0:00:25.182414 0:00:24.058991
0:00:49.349034                 test_that_multiple_keyword_specific_tokens_are_located_shuffle    1 0:00:49.349034 0:00:49.349034
0:00:49.208487                                         test_small_time_mismatches_are_ignored    1 0:00:49.208487 0:00:49.208487
0:00:46.367665                              test_that_posterior_has_lower_variance_than_prior    2 0:00:23.183832 0:00:23.088973
0:00:44.542402             test_that_surfaces_retain_their_order_when_loaded_and_saved_by_ert    2 0:00:22.271201 0:00:21.965741
0:00:42.448540                                            test_setting_env_context_during_run    4 0:00:10.582577 0:00:09.250817
0:00:41.500896      test_that_running_ies_with_different_steplength_produces_different_result    1 0:00:41.500896 0:00:41.500896
0:00:38.685962                                                        test_field_param_update    1 0:00:38.685962 0:00:38.685962
0:00:35.859027                         test_parameter_update_with_inactive_cells_xtgeo_grdecl    1 0:00:35.859027 0:00:35.859027
0:00:33.968378                                                        test_ensemble_evaluator    2 0:00:16.984189 0:00:15.878296
0:00:32.874286                   test_that_reading_summaries_returns_the_contents_of_the_file    1 0:00:32.874286 0:00:32.874286
0:00:32.687796                                                 test_shell_scripts_integration    2 0:00:16.343898 0:00:15.684761
0:00:28.670576                                                   test_direct_dark_performance    16 0:00:01.703688 0:00:00.845345
0:00:25.560582                                             test_failing_job_cli_error_message    2 0:00:12.780291 0:00:11.956501
0:00:24.579743                            test_that_setenv_sets_environment_variables_in_jobs    2 0:00:12.289872 0:00:09.581446
0:00:23.804907       test_that_the_model_warns_when_active_realizations_less_min_realizations    2 0:00:11.902453 0:00:09.710251
0:00:23.231297                                             test_batch_ctx_status_failing_jobs    2 0:00:11.615648 0:00:07.570845
0:00:20.750755                                                        test_field_param_memory    1 0:00:20.750755 0:00:20.750755
0:47:04.850058                                                                    grand total  1824 0:00:00.054200 0:00:00.000268
==================================================== test setup duration top ====================================================
total          name                                                                           num avg            min           
0:01:34.105091                                                             test_update_report    2 0:00:47.052546 0:00:00.921310
0:01:29.147664                                    test_summary_data_verify_indices_and_values    1 0:01:29.147664 0:01:29.147664
0:01:28.995839                                                             test_export_misfit    1 0:01:28.995839 0:01:28.995839
0:01:08.290464                                         test_that_different_length_values_fail    1 0:01:08.290464 0:01:08.290464
0:00:47.388024                                                     test_all_measured_snapshot    1 0:00:47.388024 0:00:47.388024
0:00:26.708200                                                               test_history_obs    1 0:00:26.708200 0:00:26.708200
0:00:16.851151                                                       test_keyword_type_checks    1 0:00:16.851151 0:00:16.851151
0:00:05.026485                                               test_write_snakeoil_runpath_file    4 0:00:01.163902 0:00:00.797033
0:00:04.530104                                 test_new_monitor_can_pick_up_where_we_left_off    1 0:00:04.530104 0:00:04.530104
0:00:04.123737                    test_that_batch_simulation_handles_invalid_suffixes_at_init    12 0:00:00.277372 0:00:00.203672
0:00:03.228336 test_dispatch_endpoint_clients_can_connect_and_monitor_can_shut_down_evaluator    1 0:00:03.228336 0:00:03.228336
0:00:03.084331                                                           test_update_snapshot    3 0:00:00.963168 0:00:00.771737
0:00:02.556538                                                          test_target_case_name    3 0:00:01.043193 0:00:00.453615
0:00:02.317450                           test_that_starting_with_invalid_key_raises_key_error    5 0:00:00.394295 0:00:00.227130
0:00:01.639664                test_write_to_runpath_produces_the_transformed_field_in_storage    1 0:00:01.639664 0:00:01.639664
0:00:01.560003                    test_that_batch_simulator_handles_invalid_suffixes_at_start    4 0:00:00.381797 0:00:00.343260
0:00:01.519671                                                             test_dying_batcher    1 0:00:01.519671 0:00:01.519671
0:00:01.403399                                                        test_get_coeffs_records    3 0:00:00.354426 0:00:00.321195
0:00:01.345010                                             test_export_runpath_star_parameter    1 0:00:01.345010 0:00:01.345010
0:00:01.231499                                      test_export_runpath_combination_parameter    1 0:00:01.231499 0:00:01.231499
0:00:01.132498                                                            test_refresh_facade    1 0:00:01.132498 0:00:01.132498
0:00:01.121405                                                   test_get_experiment_ensemble    1 0:00:01.121405 0:00:01.121405
0:00:01.102904                                                        test_gen_data_collector    1 0:00:01.102904 0:00:01.102904
0:00:01.095757                                                              test_localization    2 0:00:00.547879 0:00:00.500370
0:00:01.091641                                                                test_all_active    1 0:00:01.091641 0:00:01.091641
0:00:01.080122                                        test_ensure_multi_level_events_in_order    1 0:00:01.080122 0:00:01.080122
0:00:01.079397                                                          test_misfit_collector    1 0:00:01.079397 0:00:01.079397
0:00:00.889920                                                             test_gen_data_keys    1 0:00:00.889920 0:00:00.889920
0:00:00.882473                                            test_export_runpath_comma_parameter    1 0:00:00.882473 0:00:00.882473
0:00:00.858589                                                           test_misfit_endpoint    1 0:00:00.858589 0:00:00.858589
0:08:33.934604                                                                    grand total  1891 0:00:00.003658 0:00:00.000181
================================================== test teardown duration top ===================================================
total          name                                                                           num avg            min           
0:00:01.546544                                 test_new_monitor_can_pick_up_where_we_left_off    1 0:00:01.546544 0:00:01.546544
0:00:01.045994                                                        test_get_coeffs_records    3 0:00:00.223436 0:00:00.216639
0:00:00.311617                                                            test_get_experiment    1 0:00:00.311617 0:00:00.311617
0:00:00.264525                                                   test_get_experiment_ensemble    1 0:00:00.264525 0:00:00.264525
0:00:00.251182                                                            test_refresh_facade    1 0:00:00.251182 0:00:00.251182
0:00:00.239464                                                           test_misfit_endpoint    1 0:00:00.239464 0:00:00.239464
0:00:00.229497                                                              test_get_response    1 0:00:00.229497 0:00:00.229497
0:00:00.225696                                           test_get_responses_with_observations    1 0:00:00.225696 0:00:00.225696
0:00:00.214085                                                   test_get_record_observations    1 0:00:00.214085 0:00:00.214085
0:00:00.208582                                               test_get_experiment_observations    1 0:00:00.208582 0:00:00.208582
0:00:00.200503                                                   test_get_ensemble_parameters    1 0:00:00.200503 0:00:00.200503
0:00:00.186509                                                              test_get_ensemble    1 0:00:00.186509 0:00:00.186509
0:00:00.144616                                                         test_gen_kw_trans_func    47 0:00:00.001765 0:00:00.001038
0:00:00.085626                                                        test_custom_log_capture    25 0:00:00.003281 0:00:00.001489
0:00:00.065130                                                              test_parse_status    16 0:00:00.002938 0:00:00.001643
0:00:00.047877                      test_that_update_for_a_linear_model_works_with_rowscaling    2 0:00:00.023938 0:00:00.004924
0:00:00.046489                                  test_that_stop_on_fail_workflow_jobs_stop_ert    16 0:00:00.002350 0:00:00.000925
0:00:00.045891                   test_inter_region_summary_format_contains_in_and_out_regions    25 0:00:00.001226 0:00:00.000844
0:00:00.043891                             test_inter_region_summary_variables_are_recognized    25 0:00:00.001196 0:00:00.000941
0:00:00.040931                                                                  test_tracking    8 0:00:00.004958 0:00:00.000963
0:00:00.040418                                                test_rangestring_to_mask_errors    20 0:00:00.001067 0:00:00.000981
0:00:00.039551                                                      test_gen_kw_is_log_or_not    10 0:00:00.001743 0:00:00.001690
0:00:00.037843                                                     test_gen_kw_params_parsing    19 0:00:00.001074 0:00:00.000995
0:00:00.037756                                                       test_rangestring_to_mask    24 0:00:00.001114 0:00:00.001049
0:00:00.036888                                                   test_direct_dark_performance    48 0:00:00.000281 0:00:00.000172
0:00:00.035486                                                test_gen_kw_distribution_errors    10 0:00:00.001232 0:00:00.001130
0:00:00.029787                                              test_run_template_replace_in_file    10 0:00:00.001991 0:00:00.001744
0:00:00.029420                                                              test_verbose_flag    2 0:00:00.014710 0:00:00.013112
0:00:00.029417                                                 test_max_runtime_while_killing    1 0:00:00.029417 0:00:00.029417
0:00:00.028792                                          test_that_sampling_is_fixed_from_name    9 0:00:00.001752 0:00:00.001536
0:00:09.172072                                                                    grand total  1891 0:00:00.001412 0:00:00.000138

[eivindjahren]

As of 5e1a4223a1e1f598535666e7fcc66c14af769aeb the situation looks like this:

========================================================================================= fixture duration top =========================================================================================
total          name                                                                           num   avg            min           
0:03:39.264515                                                         _shared_snake_oil_case    26 0:00:00.000208 0:00:00.000109
0:00:17.552791                                                    poly_example_tmp_dir_shared     1 0:00:17.552791 0:00:17.552791
0:00:16.443211                                                                template_config     1 0:00:16.443211 0:00:16.443211
0:00:09.282373                                                                     ert_config    15 0:00:00.623663 0:00:00.301760
0:00:08.732345                                                         snake_oil_case_storage    24 0:00:00.333285 0:00:00.138315
0:00:07.000564                                                                 snake_oil_case    14 0:00:00.353656 0:00:00.210754
0:00:05.915693                                                              batch_sim_example    25 0:00:00.225475 0:00:00.203885
0:00:03.894245                                                                      evaluator     4 0:00:00.898614 0:00:00.480554
0:00:03.597671                                                                      poly_case    11 0:00:00.286566 0:00:00.248189
0:00:03.075036                                                            dark_storage_client    13 0:00:00.235078 0:00:00.156304
0:00:02.220284                                                                 set_ert_config    14 0:00:00.000662 0:00:00.000137
0:00:01.615702                                                                  writing_setup     6 0:00:00.253649 0:00:00.223147
0:00:01.056126                                                        snake_oil_field_example     3 0:00:00.388921 0:00:00.227985
0:00:01.005816                                                                   minimum_case     4 0:00:00.240411 0:00:00.099364
0:00:00.966346                                                    copy_snake_oil_case_storage    26 0:00:00.035270 0:00:00.016055
0:00:00.822059                                                                       tmp_path  1049 0:00:00.000651 0:00:00.000198
0:00:00.762699                                                                           data     5 0:00:00.167093 0:00:00.097548
0:00:00.597214                                                   snake_oil_export_runpath_job     2 0:00:00.298607 0:00:00.221983
0:00:00.586786                                                                       env_save  1802 0:00:00.000245 0:00:00.000084
0:00:00.412666                                                                      log_check  1802 0:00:00.000176 0:00:00.000042
0:00:00.219151                                                                no_cert_in_test  1802 0:00:00.000098 0:00:00.000039
0:00:00.214316                                                                  poly_template     1 0:00:00.214316 0:00:00.214316
0:00:00.133632                                                                        storage   184 0:00:00.000627 0:00:00.000267
0:00:00.117404                                                                 prior_ensemble    38 0:00:00.000964 0:00:00.000507
0:00:00.111540                                                                    monkeypatch  1802 0:00:00.000052 0:00:00.000020
0:00:00.106486                                                                     egrid_file    40 0:00:00.001068 0:00:00.000804
0:00:00.088444                                                                         server    10 0:00:00.005626 0:00:00.001323
0:00:00.062076                                                                 copy_poly_case    39 0:00:00.001364 0:00:00.000780
0:00:00.061668                                                               setup_mock_queue    12 0:00:00.003411 0:00:00.002918
0:00:00.035301                                                                     use_tmpdir   449 0:00:00.000073 0:00:00.000030
0:05:06.493164                                                                    grand total 12953 0:00:00.000062 0:00:00.000020
======================================================================================== test call duration top ========================================================================================
total          name                                                                           num   avg            min           
0:03:06.418580                                                          test_memory_smoothing     1 0:03:06.418580 0:03:06.418580
0:02:52.178690                                  test_that_stop_on_fail_workflow_jobs_stop_ert    16 0:00:11.145773 0:00:09.167349
0:02:49.321832                                                         test_parameter_example     2 0:01:24.660916 0:00:21.882142
0:01:42.868512                                                                  test_tracking     8 0:00:12.379418 0:00:09.181436
0:01:12.663746                  test_that_adaptive_localization_with_cutoff_0_equals_ESupdate     2 0:00:36.331873 0:00:36.193905
0:01:12.631465                   test_that_posterior_generalized_variance_increases_in_cutoff     2 0:00:36.315732 0:00:36.263338
0:01:12.320068                                                                       test_ies     2 0:00:36.160034 0:00:33.543164
0:01:04.540150                                                                    test_es_mda     2 0:00:32.270075 0:00:29.534481
0:01:04.346357                                                           TestStorage::runTest     1 0:01:04.346357 0:01:04.346357
0:00:56.211422                      test_that_prior_is_not_overwritten_in_ensemble_experiment     6 0:00:09.343042 0:00:09.270904
0:00:54.124762                                                     test_update_multiple_param     2 0:00:27.062381 0:00:26.036258
0:00:49.601187                                                 test_shell_scripts_integration     2 0:00:24.800593 0:00:15.813480
0:00:48.392188                                                      test_run_mocked_lsf_queue     2 0:00:24.196094 0:00:21.181607
0:00:46.931664                          test_surface_init_fails_during_forward_model_callback     2 0:00:23.465832 0:00:13.249322
0:00:40.774203                                            test_setting_env_context_during_run     4 0:00:10.162639 0:00:09.156919
0:00:39.055640             test_that_surfaces_retain_their_order_when_loaded_and_saved_by_ert     2 0:00:19.527820 0:00:17.585928
0:00:38.339633                                                      test_surface_param_update     1 0:00:38.339633 0:00:38.339633
0:00:36.331061            test_that_adaptive_localization_with_cutoff_1_equals_ensemble_prior     2 0:00:18.165531 0:00:18.125688
0:00:36.209547                              test_that_posterior_has_lower_variance_than_prior     2 0:00:18.104773 0:00:18.070151
0:00:33.450549                                         test_small_time_mismatches_are_ignored     1 0:00:33.450549 0:00:33.450549
0:00:33.384792                                     test_ensemble_evaluator_disable_monitoring     2 0:00:16.692396 0:00:15.841820
0:00:32.604449                                                        test_ensemble_evaluator     2 0:00:16.302224 0:00:15.308403
0:00:30.726618      test_that_running_ies_with_different_steplength_produces_different_result     1 0:00:30.726618 0:00:30.726618
0:00:28.704139                                test_that_update_works_with_failed_realizations     1 0:00:28.704139 0:00:28.704139
0:00:27.215303       test_that_the_model_warns_when_active_realizations_less_min_realizations     2 0:00:13.607652 0:00:13.438188
0:00:24.919177                 test_that_multiple_keyword_specific_tokens_are_located_shuffle     1 0:00:24.919177 0:00:24.919177
0:00:24.110859                                             test_batch_ctx_status_failing_jobs     2 0:00:12.055429 0:00:09.551931
0:00:23.323237                                                        test_field_param_update     1 0:00:23.323237 0:00:23.323237
0:00:21.338965                         test_parameter_update_with_inactive_cells_xtgeo_grdecl     1 0:00:21.338965 0:00:21.338965
0:00:20.344757                                             test_failing_job_cli_error_message     2 0:00:10.172379 0:00:09.165182
0:37:47.740378                                                                    grand total  1800 0:00:00.037106 0:00:00.000183
======================================================================================= test setup duration top ========================================================================================
total          name                                                                           num   avg            min           
0:00:44.995802                                                             test_export_misfit     1 0:00:44.995802 0:00:44.995802
0:00:43.526689                                                             test_update_report     2 0:00:21.763345 0:00:00.387689
0:00:39.883341                                    test_summary_data_verify_indices_and_values     1 0:00:39.883341 0:00:39.883341
0:00:35.710493                                         test_that_different_length_values_fail     1 0:00:35.710493 0:00:35.710493
0:00:24.022258                                                       test_gen_obs_and_summary     1 0:00:24.022258 0:00:24.022258
0:00:20.534020                                                               test_history_obs     1 0:00:20.534020 0:00:20.534020
0:00:13.464383                                                       test_keyword_type_checks     1 0:00:13.464383 0:00:13.464383
0:00:02.879384                    test_that_batch_simulation_handles_invalid_suffixes_at_init    12 0:00:00.222103 0:00:00.205585
0:00:01.871878                                               test_write_snakeoil_runpath_file     4 0:00:00.358882 0:00:00.343301
0:00:01.618062 test_dispatch_endpoint_clients_can_connect_and_monitor_can_shut_down_evaluator     1 0:00:01.618062 0:00:01.618062
0:00:01.355430                                                          test_target_case_name     3 0:00:00.450539 0:00:00.268008
0:00:01.282364                                                 test_data_fetching_missing_key     1 0:00:01.282364 0:00:01.282364
0:00:01.165731                           test_that_starting_with_invalid_key_raises_key_error     5 0:00:00.229282 0:00:00.212646
0:00:01.133975                                                             test_migrate_field     1 0:00:01.133975 0:00:01.133975
0:00:01.082152                                                           test_update_snapshot     3 0:00:00.355245 0:00:00.289247
0:00:00.958169                                                        test_all_data_type_keys     2 0:00:00.479084 0:00:00.012070
0:00:00.946581                                 test_new_monitor_can_pick_up_where_we_left_off     1 0:00:00.946581 0:00:00.946581
0:00:00.939430                    test_that_batch_simulator_handles_invalid_suffixes_at_start     4 0:00:00.229141 0:00:00.214829
0:00:00.884200                                                               test_summary_obs     1 0:00:00.884200 0:00:00.884200
0:00:00.853254                                        test_ensure_multi_level_events_in_order     1 0:00:00.853254 0:00:00.853254
0:00:00.851512                                                                   test_gen_obs     1 0:00:00.851512 0:00:00.851512
0:00:00.709957                                                              test_localization     2 0:00:00.354978 0:00:00.327774
0:00:00.694765                                                          test_observation_keys     1 0:00:00.694765 0:00:00.694765
0:00:00.691956                                                        test_get_coeffs_records     3 0:00:00.231340 0:00:00.223623
0:00:00.673803                                                         test_migration_failure     3 0:00:00.008565 0:00:00.008384
0:00:00.571407                                                              test_migrate_case     3 0:00:00.008721 0:00:00.007583
0:00:00.547846                                                             test_gen_data_keys     1 0:00:00.547846 0:00:00.547846
0:00:00.542436                                                           test_migrate_surface     1 0:00:00.542436 0:00:00.542436
0:00:00.509231                                                 test_batch_simulation_suffixes     2 0:00:00.254615 0:00:00.233495
0:00:00.495936                                                          test_batch_simulation     2 0:00:00.247968 0:00:00.219645
0:04:27.490159                                                                    grand total  1867 0:00:00.002513 0:00:00.000126
====================================================================================== test teardown duration top ======================================================================================
total          name                                                                           num   avg            min           
0:00:01.656054                                 test_new_monitor_can_pick_up_where_we_left_off     1 0:00:01.656054 0:00:01.656054
0:00:00.561265                                                        test_get_coeffs_records     3 0:00:00.165436 0:00:00.164206
0:00:00.276311                                                            test_get_experiment     1 0:00:00.276311 0:00:00.276311
0:00:00.169160                                                              test_get_ensemble     1 0:00:00.169160 0:00:00.169160
0:00:00.168864                                                   test_get_ensemble_parameters     1 0:00:00.168864 0:00:00.168864
0:00:00.167779                                                   test_get_experiment_ensemble     1 0:00:00.167779 0:00:00.167779
0:00:00.163719                                           test_get_responses_with_observations     1 0:00:00.163719 0:00:00.163719
0:00:00.161120                                               test_get_experiment_observations     1 0:00:00.161120 0:00:00.161120
0:00:00.161084                                                              test_get_response     1 0:00:00.161084 0:00:00.161084
0:00:00.159767                                                           test_misfit_endpoint     1 0:00:00.159767 0:00:00.159767
0:00:00.157163                                                   test_get_record_observations     1 0:00:00.157163 0:00:00.157163
0:00:00.134415                                                            test_refresh_facade     1 0:00:00.134415 0:00:00.134415
0:00:00.075972                                                       test_rangestring_to_mask    24 0:00:00.001976 0:00:00.001787
0:00:00.069245                                                test_rangestring_to_list_errors    10 0:00:00.002549 0:00:00.001631
0:00:00.057911                                                test_rangestring_to_mask_errors    20 0:00:00.002093 0:00:00.001439
0:00:00.049432                                                         test_gen_kw_trans_func    47 0:00:00.001022 0:00:00.000869
0:00:00.046144                                          test_analysis_config_min_realizations    12 0:00:00.001068 0:00:00.000737
0:00:00.033606                                                        test_custom_log_capture    25 0:00:00.001155 0:00:00.001073
0:00:00.025107                                                   test_direct_dark_performance    48 0:00:00.000242 0:00:00.000118
0:00:00.023862                                          test_that_sampling_is_fixed_from_name     9 0:00:00.002813 0:00:00.001789
0:00:00.022418                                                       test_mask_to_rangestring    12 0:00:00.001685 0:00:00.001622
0:00:00.021903                                  test_that_stop_on_fail_workflow_jobs_stop_ert    16 0:00:00.001372 0:00:00.000630
0:00:00.021342                                                    test_activerange_initerrors     6 0:00:00.002140 0:00:00.001921
0:00:00.020990                             test_inter_region_summary_variables_are_recognized    25 0:00:00.000812 0:00:00.000753
0:00:00.020553                   test_inter_region_summary_format_contains_in_and_out_regions    25 0:00:00.000814 0:00:00.000759
0:00:00.018147                                                     test_gen_kw_params_parsing    19 0:00:00.000920 0:00:00.000865
0:00:00.017427                                             test_batch_ctx_status_failing_jobs     2 0:00:00.008713 0:00:00.002678
0:00:00.017359                    test_that_loading_summary_obs_with_days_is_within_tolerance    16 0:00:00.001073 0:00:00.000932
0:00:00.016189                                                             test_surface_param     5 0:00:00.003239 0:00:00.003077
0:00:00.016026                                                       test_rangestring_to_list     8 0:00:00.001707 0:00:00.001233
0:00:06.430711                                                                    grand total  1867 0:00:00.001000 0:00:00.000118

[eivindjahren]

Situation as of 942555d4ad6834c2b66741354fc73f18c7f94e11:

=========================================================================================== fixture duration top ===========================================================================================
total          name                                                                           num   avg            min           
0:04:21.624848                                                         _shared_snake_oil_case    26 0:00:00.000215 0:00:00.000126
0:00:35.251494                                                    poly_example_tmp_dir_shared     2 0:00:17.625747 0:00:15.899171
0:00:14.646384                                                                template_config     1 0:00:14.646384 0:00:14.646384
0:00:07.927343                                                         snake_oil_case_storage    24 0:00:00.327873 0:00:00.140673
0:00:07.058581                                                              batch_sim_example    25 0:00:00.276035 0:00:00.143935
0:00:06.148988                                                                 snake_oil_case    14 0:00:00.401902 0:00:00.146367
0:00:04.622703                                                                      evaluator     4 0:00:01.218940 0:00:00.717864
0:00:04.244515                                                                      poly_case    11 0:00:00.355248 0:00:00.239973
0:00:03.145461                                                            dark_storage_client    13 0:00:00.234975 0:00:00.129189
0:00:02.995922                                                                  writing_setup     6 0:00:00.494552 0:00:00.471026
0:00:02.685765                                                                     ert_config    13 0:00:00.213017 0:00:00.142113
0:00:01.532972                                                                   minimum_case     4 0:00:00.284979 0:00:00.229072
0:00:01.500792                                                                 set_ert_config    13 0:00:00.000952 0:00:00.000050
0:00:01.158269                                                        snake_oil_field_example     3 0:00:00.475634 0:00:00.160380
0:00:00.943875                                                   snake_oil_export_runpath_job     2 0:00:00.471938 0:00:00.335730
0:00:00.920394                                                    copy_snake_oil_case_storage    26 0:00:00.035552 0:00:00.019230
0:00:00.853735                                                                       tmp_path  1050 0:00:00.000683 0:00:00.000206
0:00:00.516388                                                                       env_save  1807 0:00:00.000246 0:00:00.000087
0:00:00.379278                                                                      log_check  1807 0:00:00.000177 0:00:00.000075
0:00:00.283924                                                                  poly_template     1 0:00:00.283924 0:00:00.283924
0:00:00.194598                                                                no_cert_in_test  1807 0:00:00.000097 0:00:00.000041
0:00:00.178803                                                                           data     4 0:00:00.043728 0:00:00.036918
0:00:00.157131                                                                        storage   184 0:00:00.000870 0:00:00.000413
0:00:00.103621                                                                 prior_ensemble    38 0:00:00.001161 0:00:00.000566
0:00:00.100294                                                                    monkeypatch  1807 0:00:00.000053 0:00:00.000020
0:00:00.073462                                                                 copy_poly_case    39 0:00:00.001630 0:00:00.000861
0:00:00.053463                                                                         server    10 0:00:00.000998 0:00:00.000741
0:00:00.052410                                                              mocked_valid_file    35 0:00:00.000277 0:00:00.000201
0:00:00.039021                                                                            api     7 0:00:00.004285 0:00:00.004036
0:00:00.038871                                                                     egrid_file    40 0:00:00.001028 0:00:00.000610
0:05:59.964806                                                                    grand total 12990 0:00:00.000059 0:00:00.000020
========================================================================================== test call duration top ==========================================================================================
total          name                                                                           num   avg            min           
0:03:04.226487                                                          test_memory_smoothing     1 0:03:04.226487 0:03:04.226487
0:02:56.752808                                  test_that_stop_on_fail_workflow_jobs_stop_ert    16 0:00:11.148732 0:00:09.140415
0:02:37.212042                                                         test_parameter_example     1 0:02:37.212042 0:02:37.212042
0:01:55.746551                                                                  test_tracking     8 0:00:14.409473 0:00:09.338402
0:01:23.229596                   test_that_posterior_generalized_variance_increases_in_cutoff     2 0:00:41.614798 0:00:40.558807
0:01:21.160251                  test_that_adaptive_localization_with_cutoff_0_equals_ESupdate     2 0:00:40.580126 0:00:40.566907
0:01:12.488496                                                     test_update_multiple_param     2 0:00:36.244248 0:00:28.008270
0:01:12.316713                                                           TestStorage::runTest     1 0:01:12.316713 0:01:12.316713
0:01:10.198557                                                                       test_ies     2 0:00:35.099278 0:00:33.936420
0:00:58.028487                      test_that_prior_is_not_overwritten_in_ensemble_experiment     6 0:00:09.345592 0:00:09.276292
0:00:57.077127                                                                    test_es_mda     2 0:00:28.538564 0:00:27.562017
0:00:55.193532                          test_surface_init_fails_during_forward_model_callback     2 0:00:27.596766 0:00:15.537972
0:00:47.069266             test_that_surfaces_retain_their_order_when_loaded_and_saved_by_ert     2 0:00:23.534633 0:00:23.492305
0:00:46.804371                                                      test_run_mocked_lsf_queue     2 0:00:23.402186 0:00:21.535685
0:00:43.619446                                     test_ensemble_evaluator_disable_monitoring     2 0:00:21.809723 0:00:17.579454
0:00:42.391850                                                        test_field_param_memory     1 0:00:42.391850 0:00:42.391850
0:00:40.942088                                            test_setting_env_context_during_run     4 0:00:10.237305 0:00:09.189664
0:00:40.759176                              test_that_posterior_has_lower_variance_than_prior     2 0:00:20.379588 0:00:18.532383
0:00:39.807443                                         test_small_time_mismatches_are_ignored     1 0:00:39.807443 0:00:39.807443
0:00:36.200991            test_that_adaptive_localization_with_cutoff_1_equals_ensemble_prior     2 0:00:18.100495 0:00:18.065498
0:00:34.081428                                test_that_update_works_with_failed_realizations     1 0:00:34.081428 0:00:34.081428
0:00:32.771738                                                        test_ensemble_evaluator     2 0:00:16.385869 0:00:15.309176
0:00:31.091454      test_that_running_ies_with_different_steplength_produces_different_result     1 0:00:31.091454 0:00:31.091454
0:00:30.014282                                                 test_shell_scripts_integration     2 0:00:15.007141 0:00:14.309979
0:00:28.637747                 test_that_multiple_keyword_specific_tokens_are_located_shuffle     1 0:00:28.637747 0:00:28.637747
0:00:23.439922                         test_parameter_update_with_inactive_cells_xtgeo_grdecl     1 0:00:23.439922 0:00:23.439922
0:00:21.933860                                             test_batch_ctx_status_failing_jobs     2 0:00:10.966930 0:00:07.334440
0:00:21.617061                                                        test_field_param_update     1 0:00:21.617061 0:00:21.617061
0:00:20.726870       test_that_the_model_warns_when_active_realizations_less_min_realizations     2 0:00:10.363435 0:00:09.354550
0:00:20.267489                                             test_failing_job_cli_error_message     2 0:00:10.133745 0:00:09.143004
0:38:31.559799                                                                    grand total  1805 0:00:00.029314 0:00:00.000191
========================================================================================= test setup duration top =========================================================================================
total          name                                                                           num   avg            min           
0:00:57.424519                                                             test_update_report     2 0:00:28.712259 0:00:00.354820
0:00:55.261893                                                             test_export_misfit     1 0:00:55.261893 0:00:55.261893
0:00:46.561019                                    test_summary_data_verify_indices_and_values     1 0:00:46.561019 0:00:46.561019
0:00:43.651351                                         test_that_different_length_values_fail     1 0:00:43.651351 0:00:43.651351
0:00:28.302069                                                     test_all_measured_snapshot     1 0:00:28.302069 0:00:28.302069
0:00:17.802243                                                               test_history_obs     1 0:00:17.802243 0:00:17.802243
0:00:15.408968                                                       test_keyword_type_checks     1 0:00:15.408968 0:00:15.408968
0:00:03.244813                    test_that_batch_simulation_handles_invalid_suffixes_at_init    12 0:00:00.275071 0:00:00.146237
0:00:01.781786                                               test_write_snakeoil_runpath_file     4 0:00:00.473207 0:00:00.331566
0:00:01.623495                    test_that_batch_simulator_handles_invalid_suffixes_at_start     4 0:00:00.398084 0:00:00.290512
0:00:01.583257                                                           test_update_snapshot     3 0:00:00.466932 0:00:00.445202
0:00:01.468691                                                             test_dying_batcher     1 0:00:01.468691 0:00:01.468691
0:00:01.352307                           test_that_starting_with_invalid_key_raises_key_error     5 0:00:00.272321 0:00:00.236049
0:00:01.339122                                        test_ensure_multi_level_events_in_order     1 0:00:01.339122 0:00:01.339122
0:00:01.101976 test_dispatch_endpoint_clients_can_connect_and_monitor_can_shut_down_evaluator     1 0:00:01.101976 0:00:01.101976
0:00:01.019038                                                          test_observation_keys     1 0:00:01.019038 0:00:01.019038
0:00:00.978673                                                          test_target_case_name     3 0:00:00.357649 0:00:00.242347
0:00:00.925571                                                        test_all_data_type_keys     2 0:00:00.462786 0:00:00.009975
0:00:00.748085                                                                      test_init     2 0:00:00.374042 0:00:00.011546
0:00:00.719567                                 test_new_monitor_can_pick_up_where_we_left_off     1 0:00:00.719567 0:00:00.719567
0:00:00.610386                                       test_export_runpath_number_of_iterations     1 0:00:00.610386 0:00:00.610386
0:00:00.584959                                          test_setup_multiple_data_assimilation     1 0:00:00.584959 0:00:00.584959
0:00:00.557946                                                        test_get_coeffs_records     3 0:00:00.190312 0:00:00.131097
0:00:00.548444                                         test_setup_iterative_ensemble_smoother     1 0:00:00.548444 0:00:00.548444
0:00:00.543176                                                              test_localization     2 0:00:00.271588 0:00:00.241673
0:00:00.541983                                           test_gen_obs_and_summary_index_range     1 0:00:00.541983 0:00:00.541983
0:00:00.540758                                              test_observation_keys_missing_key     1 0:00:00.540758 0:00:00.540758
0:00:00.540169                                             test_export_runpath_star_parameter     1 0:00:00.540169 0:00:00.540169
0:00:00.528398                                                     test_setup_single_test_run     1 0:00:00.528398 0:00:00.528398
0:00:00.528200                                                              test_summary_keys     1 0:00:00.528200 0:00:00.528200
0:05:10.146215                                                                    grand total  1872 0:00:00.002557 0:00:00.000114
======================================================================================== test teardown duration top =======================================================================================
total          name                                                                           num   avg            min           
0:00:01.649724                                 test_new_monitor_can_pick_up_where_we_left_off     1 0:00:01.649724 0:00:01.649724
0:00:00.505511                                                        test_get_coeffs_records     3 0:00:00.157521 0:00:00.145825
0:00:00.261257                                                   test_get_experiment_ensemble     1 0:00:00.261257 0:00:00.261257
0:00:00.252643                                                            test_refresh_facade     1 0:00:00.252643 0:00:00.252643
0:00:00.231927                                                            test_get_experiment     1 0:00:00.231927 0:00:00.231927
0:00:00.226406                                                              test_get_ensemble     1 0:00:00.226406 0:00:00.226406
0:00:00.212274                                               test_get_experiment_observations     1 0:00:00.212274 0:00:00.212274
0:00:00.188814                                                              test_get_response     1 0:00:00.188814 0:00:00.188814
0:00:00.185111                                                   test_get_ensemble_parameters     1 0:00:00.185111 0:00:00.185111
0:00:00.184542                                           test_get_responses_with_observations     1 0:00:00.184542 0:00:00.184542
0:00:00.150355                                                   test_get_record_observations     1 0:00:00.150355 0:00:00.150355
0:00:00.137224                                                           test_misfit_endpoint     1 0:00:00.137224 0:00:00.137224
0:00:00.074421                                                          test_workflow_configs     3 0:00:00.018819 0:00:00.000753
0:00:00.054235                                                         test_gen_kw_trans_func    47 0:00:00.001126 0:00:00.001000
0:00:00.045494                                                       test_rangestring_to_mask    24 0:00:00.000987 0:00:00.000664
0:00:00.044196                                          test_analysis_config_min_realizations    12 0:00:00.002846 0:00:00.000840
0:00:00.040040                                                        test_custom_log_capture    25 0:00:00.001323 0:00:00.000655
0:00:00.039499                                                                  test_tracking     8 0:00:00.002125 0:00:00.001809
0:00:00.027667                                                test_rangestring_to_mask_errors    20 0:00:00.000966 0:00:00.000631
0:00:00.027188                                                test_rangestring_to_list_errors    10 0:00:00.001225 0:00:00.000795
0:00:00.024985                             test_inter_region_summary_variables_are_recognized    25 0:00:00.000909 0:00:00.000846
0:00:00.024809                                  test_that_stop_on_fail_workflow_jobs_stop_ert    16 0:00:00.001812 0:00:00.000674
0:00:00.024197                   test_inter_region_summary_format_contains_in_and_out_regions    25 0:00:00.000879 0:00:00.000471
0:00:00.023175                    test_that_loading_summary_obs_with_days_is_within_tolerance    16 0:00:00.001199 0:00:00.001103
0:00:00.021664                                                   test_direct_dark_performance    48 0:00:00.000287 0:00:00.000107
0:00:00.021392                                     test_that_parsing_workflows_gives_expected     1 0:00:00.021392 0:00:00.021392
0:00:00.021359                                                     test_gen_kw_params_parsing    19 0:00:00.001019 0:00:00.000946
0:00:00.019930                                                              test_parse_status    16 0:00:00.001296 0:00:00.001007
0:00:00.018679                    test_that_batch_simulation_handles_invalid_suffixes_at_init    12 0:00:00.001214 0:00:00.000701
0:00:00.017492                                                   test_check_if_runpath_exists    14 0:00:00.001192 0:00:00.001018
0:00:06.672999