search

fatiando / verde

Processing and gridding spatial data, machine-learning style
https://www.fatiando.org/verde
BSD 3-Clause "New" or "Revised" License
602 stars 72 forks source link

Tests failing on Python 3.8 and Dask 2021.3.0 #310

Closed santisoler closed 3 years ago

santisoler commented 3 years ago

Description of the problem

Some tests that run delayed operations fail on Python 3.8 and Dask 2021.3.0.

Full code that generated the error

make test

Full error message

# Run a tmp folder to make sure the tests are run on the installed version
mkdir -p tmp-test-dir-with-unique-name
cd tmp-test-dir-with-unique-name; MPLBACKEND='agg' pytest --cov-config=../.coveragerc --cov-report=term-missing --cov=verde --doctest-modules -v --pyargs verde
=========================================================================================================== test session starts ============================================================================================================
platform linux -- Python 3.8.8, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /home/santi/.anaconda3/envs/verde-3.8/bin/python
cachedir: .pytest_cache
Matplotlib: 3.3.4
Freetype: 2.10.4
rootdir: /home/santi/git/verde
plugins: cov-2.11.1, mpl-0.12
collected 182 items                                                                                                                                                                                                                        

../verde/coordinates.py::verde.coordinates.block_split PASSED                                                                                                                                                                        [  0%]
../verde/coordinates.py::verde.coordinates.expanding_window PASSED                                                                                                                                                                   [  1%]
../verde/coordinates.py::verde.coordinates.get_region PASSED                                                                                                                                                                         [  1%]
../verde/coordinates.py::verde.coordinates.grid_coordinates PASSED                                                                                                                                                                   [  2%]
../verde/coordinates.py::verde.coordinates.inside PASSED                                                                                                                                                                             [  2%]
../verde/coordinates.py::verde.coordinates.longitude_continuity PASSED                                                                                                                                                               [  3%]
../verde/coordinates.py::verde.coordinates.pad_region PASSED                                                                                                                                                                         [  3%]
../verde/coordinates.py::verde.coordinates.profile_coordinates PASSED                                                                                                                                                                [  4%]
../verde/coordinates.py::verde.coordinates.rolling_window PASSED                                                                                                                                                                     [  4%]
../verde/coordinates.py::verde.coordinates.scatter_points PASSED                                                                                                                                                                     [  5%]
../verde/coordinates.py::verde.coordinates.shape_to_spacing PASSED                                                                                                                                                                   [  6%]
../verde/distances.py::verde.distances.median_distance PASSED                                                                                                                                                                        [  6%]
../verde/mask.py::verde.mask.convexhull_mask PASSED                                                                                                                                                                                  [  7%]
../verde/mask.py::verde.mask.distance_mask PASSED                                                                                                                                                                                    [  7%]
../verde/model_selection.py::verde.model_selection.BlockKFold PASSED                                                                                                                                                                 [  8%]
../verde/model_selection.py::verde.model_selection.BlockShuffleSplit PASSED                                                                                                                                                          [  8%]
../verde/model_selection.py::verde.model_selection.cross_val_score FAILED                                                                                                                                                            [  9%]
../verde/model_selection.py::verde.model_selection.select PASSED                                                                                                                                                                     [  9%]
../verde/model_selection.py::verde.model_selection.train_test_split PASSED                                                                                                                                                           [ 10%]
../verde/projections.py::verde.projections.project_region PASSED                                                                                                                                                                     [ 10%]
../verde/trend.py::verde.trend.Trend PASSED                                                                                                                                                                                          [ 11%]
../verde/trend.py::verde.trend.Trend.jacobian PASSED                                                                                                                                                                                 [ 12%]
../verde/trend.py::verde.trend.polynomial_power_combinations PASSED                                                                                                                                                                  [ 12%]
../verde/utils.py::verde.utils.grid_to_table PASSED                                                                                                                                                                                  [ 13%]
../verde/utils.py::verde.utils.make_xarray_grid PASSED                                                                                                                                                                               [ 13%]
../verde/utils.py::verde.utils.maxabs PASSED                                                                                                                                                                                         [ 14%]
../verde/utils.py::verde.utils.partition_by_sum PASSED                                                                                                                                                                               [ 14%]
../verde/utils.py::verde.utils.variance_to_weights PASSED                                                                                                                                                                            [ 15%]
../verde/base/base_classes.py::verde.base.base_classes.BaseGridder PASSED                                                                                                                                                            [ 15%]
../verde/base/base_classes.py::verde.base.base_classes.BaseGridder._get_data_names PASSED                                                                                                                                            [ 16%]
../verde/base/base_classes.py::verde.base.base_classes.BaseGridder._get_extra_coords_names PASSED                                                                                                                                    [ 17%]
../verde/base/base_classes.py::verde.base.base_classes.project_coordinates PASSED                                                                                                                                                    [ 17%]
../verde/base/utils.py::verde.base.utils.DummyEstimator PASSED                                                                                                                                                                       [ 18%]
../verde/base/utils.py::verde.base.utils.check_data PASSED                                                                                                                                                                           [ 18%]
../verde/base/utils.py::verde.base.utils.check_data_names PASSED                                                                                                                                                                     [ 19%]
../verde/base/utils.py::verde.base.utils.check_extra_coords_names PASSED                                                                                                                                                             [ 19%]
../verde/base/utils.py::verde.base.utils.n_1d_arrays PASSED                                                                                                                                                                          [ 20%]
../verde/datasets/synthetic.py::verde.datasets.synthetic.CheckerBoard PASSED                                                                                                                                                         [ 20%]
../verde/tests/test_base.py::test_check_coordinates PASSED                                                                                                                                                                           [ 21%]
../verde/tests/test_base.py::test_get_dims PASSED                                                                                                                                                                                    [ 21%]
../verde/tests/test_base.py::test_get_data_names PASSED                                                                                                                                                                              [ 22%]
../verde/tests/test_base.py::test_get_data_names_fails PASSED                                                                                                                                                                        [ 23%]
../verde/tests/test_base.py::test_get_instance_region PASSED                                                                                                                                                                         [ 23%]
../verde/tests/test_base.py::test_basegridder PASSED                                                                                                                                                                                 [ 24%]
../verde/tests/test_base.py::test_basegridder_data_names PASSED                                                                                                                                                                      [ 24%]
../verde/tests/test_base.py::test_basegridder_projection PASSED                                                                                                                                                                      [ 25%]
../verde/tests/test_base.py::test_basegridder_extra_coords PASSED                                                                                                                                                                    [ 25%]
../verde/tests/test_base.py::test_basegridder_projection_multiple_coordinates PASSED                                                                                                                                                 [ 26%]
../verde/tests/test_base.py::test_check_fit_input PASSED                                                                                                                                                                             [ 26%]
../verde/tests/test_base.py::test_check_fit_input_fails_coordinates PASSED                                                                                                                                                           [ 27%]
../verde/tests/test_base.py::test_check_fit_input_fails_weights PASSED                                                                                                                                                               [ 28%]
../verde/tests/test_base.py::test_baseblockedcrossvalidator_n_splits PASSED                                                                                                                                                          [ 28%]
../verde/tests/test_base.py::test_baseblockedcrossvalidator_fails_spacing_shape PASSED                                                                                                                                               [ 29%]
../verde/tests/test_base.py::test_baseblockedcrossvalidator_fails_data_shape PASSED                                                                                                                                                  [ 29%]
../verde/tests/test_base.py::test_least_squares_copy_jacobian PASSED                                                                                                                                                                 [ 30%]
../verde/tests/test_blockreduce.py::test_block_reduce PASSED                                                                                                                                                                         [ 30%]
../verde/tests/test_blockreduce.py::test_block_reduce_shape PASSED                                                                                                                                                                   [ 31%]
../verde/tests/test_blockreduce.py::test_block_reduce_scatter PASSED                                                                                                                                                                 [ 31%]
../verde/tests/test_blockreduce.py::test_block_reduce_weights PASSED                                                                                                                                                                 [ 32%]
../verde/tests/test_blockreduce.py::test_block_reduce_drop_coords PASSED                                                                                                                                                             [ 32%]
../verde/tests/test_blockreduce.py::test_block_reduce_multiple_coordinates PASSED                                                                                                                                                    [ 33%]
../verde/tests/test_blockreduce.py::test_block_reduce_scatter_multiple_coordinates PASSED                                                                                                                                            [ 34%]
../verde/tests/test_blockreduce.py::test_block_reduce_multiple_components PASSED                                                                                                                                                     [ 34%]
../verde/tests/test_blockreduce.py::test_block_reduce_multiple_weights PASSED                                                                                                                                                        [ 35%]
../verde/tests/test_blockreduce.py::test_blockmean_noweights PASSED                                                                                                                                                                  [ 35%]
../verde/tests/test_blockreduce.py::test_blockmean_noweights_multiple_components PASSED                                                                                                                                              [ 36%]
../verde/tests/test_blockreduce.py::test_blockmean_noweights_table PASSED                                                                                                                                                            [ 36%]
../verde/tests/test_blockreduce.py::test_blockmean_uncertainty_weights PASSED                                                                                                                                                        [ 37%]
../verde/tests/test_blockreduce.py::test_blockmean_variance_weights PASSED                                                                                                                                                           [ 37%]
../verde/tests/test_chain.py::test_chain PASSED                                                                                                                                                                                      [ 38%]
../verde/tests/test_chain.py::test_chain_double PASSED                                                                                                                                                                               [ 39%]
../verde/tests/test_chain.py::test_chain_vector PASSED                                                                                                                                                                               [ 39%]
../verde/tests/test_coordinates.py::test_rolling_window_invalid_coordinate_shapes PASSED                                                                                                                                             [ 40%]
../verde/tests/test_coordinates.py::test_rolling_window_empty PASSED                                                                                                                                                                 [ 40%]
../verde/tests/test_coordinates.py::test_rolling_window_warnings PASSED                                                                                                                                                              [ 41%]
../verde/tests/test_coordinates.py::test_rolling_window_no_shape_or_spacing PASSED                                                                                                                                                   [ 41%]
../verde/tests/test_coordinates.py::test_rolling_window_oversized_window PASSED                                                                                                                                                      [ 42%]
../verde/tests/test_coordinates.py::test_spacing_to_shape PASSED                                                                                                                                                                     [ 42%]
../verde/tests/test_coordinates.py::test_spacing_to_shape_fails PASSED                                                                                                                                                               [ 43%]
../verde/tests/test_coordinates.py::test_grid_coordinates_fails PASSED                                                                                                                                                               [ 43%]
../verde/tests/test_coordinates.py::test_check_region PASSED                                                                                                                                                                         [ 44%]
../verde/tests/test_coordinates.py::test_profile_coordinates_fails PASSED                                                                                                                                                            [ 45%]
../verde/tests/test_coordinates.py::test_longitude_continuity PASSED                                                                                                                                                                 [ 45%]
../verde/tests/test_coordinates.py::test_invalid_geographic_region PASSED                                                                                                                                                            [ 46%]
../verde/tests/test_coordinates.py::test_invalid_geographic_coordinates PASSED                                                                                                                                                       [ 46%]
../verde/tests/test_datasets.py::test_datasets_locate PASSED                                                                                                                                                                         [ 47%]
../verde/tests/test_datasets.py::test_fetch_texas_wind PASSED                                                                                                                                                                        [ 47%]
../verde/tests/test_datasets.py::test_setup_texas_wind PASSED                                                                                                                                                                        [ 48%]
../verde/tests/test_datasets.py::test_fetch_baja_bathymetry PASSED                                                                                                                                                                   [ 48%]
../verde/tests/test_datasets.py::test_setup_baja_bathymetry PASSED                                                                                                                                                                   [ 49%]
../verde/tests/test_datasets.py::test_fetch_rio_magnetic PASSED                                                                                                                                                                      [ 50%]
../verde/tests/test_datasets.py::test_setup_rio_magnetic PASSED                                                                                                                                                                      [ 50%]
../verde/tests/test_datasets.py::test_fetch_california_gps PASSED                                                                                                                                                                    [ 51%]
../verde/tests/test_datasets.py::test_setup_california_gps PASSED                                                                                                                                                                    [ 51%]
../verde/tests/test_datasets.py::test_setup_cartopy_backward PASSED                                                                                                                                                                  [ 52%]
../verde/tests/test_distances.py::test_distance_nearest PASSED                                                                                                                                                                       [ 52%]
../verde/tests/test_distances.py::test_distance_k_nearest PASSED                                                                                                                                                                     [ 53%]
../verde/tests/test_distances.py::test_distance_nearest_projection PASSED                                                                                                                                                            [ 53%]
../verde/tests/test_io.py::test_load_surfer_header PASSED                                                                                                                                                                            [ 54%]
../verde/tests/test_io.py::test_load_surfer PASSED                                                                                                                                                                                   [ 54%]
../verde/tests/test_io.py::test_load_surfer_nans PASSED                                                                                                                                                                              [ 55%]
../verde/tests/test_io.py::test_load_surfer_file PASSED                                                                                                                                                                              [ 56%]
../verde/tests/test_io.py::test_load_surfer_integrity PASSED                                                                                                                                                                         [ 56%]
../verde/tests/test_mask.py::test_convexhull_mask PASSED                                                                                                                                                                             [ 57%]
../verde/tests/test_mask.py::test_convexhull_mask_projection PASSED                                                                                                                                                                  [ 57%]
../verde/tests/test_mask.py::test_distance_mask PASSED                                                                                                                                                                               [ 58%]
../verde/tests/test_mask.py::test_distance_mask_projection PASSED                                                                                                                                                                    [ 58%]
../verde/tests/test_mask.py::test_distance_mask_grid PASSED                                                                                                                                                                          [ 59%]
../verde/tests/test_mask.py::test_distance_mask_missing_args PASSED                                                                                                                                                                  [ 59%]
../verde/tests/test_mask.py::test_distance_mask_wrong_shapes PASSED                                                                                                                                                                  [ 60%]
../verde/tests/test_minimal.py::test_minimal_integration_2d_gps PASSED                                                                                                                                                               [ 60%]
../verde/tests/test_model_selection.py::test_cross_val_score[none] PASSED                                                                                                                                                            [ 61%]
../verde/tests/test_model_selection.py::test_cross_val_score[R2] PASSED                                                                                                                                                              [ 62%]
../verde/tests/test_model_selection.py::test_cross_val_score[MSE] PASSED                                                                                                                                                             [ 62%]
../verde/tests/test_model_selection.py::test_cross_val_score_vector[none] PASSED                                                                                                                                                     [ 63%]
../verde/tests/test_model_selection.py::test_cross_val_score_vector[R2] PASSED                                                                                                                                                       [ 63%]
../verde/tests/test_model_selection.py::test_cross_val_score_vector[MSE] PASSED                                                                                                                                                      [ 64%]
../verde/tests/test_model_selection.py::test_cross_val_score_client PASSED                                                                                                                                                           [ 64%]
../verde/tests/test_model_selection.py::test_blockshufflesplit_fails_balancing PASSED                                                                                                                                                [ 65%]
../verde/tests/test_model_selection.py::test_blockshufflesplit_balancing[0.1] PASSED                                                                                                                                                 [ 65%]
../verde/tests/test_model_selection.py::test_blockshufflesplit_balancing[0.2] PASSED                                                                                                                                                 [ 66%]
../verde/tests/test_model_selection.py::test_blockshufflesplit_balancing[0.3] PASSED                                                                                                                                                 [ 67%]
../verde/tests/test_model_selection.py::test_blockshufflesplit_balancing[0.4] PASSED                                                                                                                                                 [ 67%]
../verde/tests/test_model_selection.py::test_blockshufflesplit_balancing[0.5] PASSED                                                                                                                                                 [ 68%]
../verde/tests/test_model_selection.py::test_blockshufflesplit_balancing[0.6] PASSED                                                                                                                                                 [ 68%]
../verde/tests/test_model_selection.py::test_blockshufflesplit_balancing[0.9] PASSED                                                                                                                                                 [ 69%]
../verde/tests/test_model_selection.py::test_blockkfold_fails_n_splits_too_small PASSED                                                                                                                                              [ 69%]
../verde/tests/test_model_selection.py::test_blockkfold_fails_n_splits_too_large PASSED                                                                                                                                              [ 70%]
../verde/tests/test_model_selection.py::test_blockkfold_cant_balance PASSED                                                                                                                                                          [ 70%]
../verde/tests/test_projections.py::test_project_grid[nearest] PASSED                                                                                                                                                                [ 71%]
../verde/tests/test_projections.py::test_project_grid[linear] PASSED                                                                                                                                                                 [ 71%]
../verde/tests/test_projections.py::test_project_grid[cubic] PASSED                                                                                                                                                                  [ 72%]
../verde/tests/test_projections.py::test_project_grid[gridder] PASSED                                                                                                                                                                [ 73%]
../verde/tests/test_projections.py::test_project_grid_name PASSED                                                                                                                                                                    [ 73%]
../verde/tests/test_projections.py::test_project_grid_antialias[True] PASSED                                                                                                                                                         [ 74%]
../verde/tests/test_projections.py::test_project_grid_antialias[False] PASSED                                                                                                                                                        [ 74%]
../verde/tests/test_projections.py::test_project_grid_fails_dataset PASSED                                                                                                                                                           [ 75%]
../verde/tests/test_projections.py::test_project_grid_fails_dimensions[1] PASSED                                                                                                                                                     [ 75%]
../verde/tests/test_projections.py::test_project_grid_fails_dimensions[3] PASSED                                                                                                                                                     [ 76%]
../verde/tests/test_scipy.py::test_scipy_gridder_same_points PASSED                                                                                                                                                                  [ 76%]
../verde/tests/test_scipy.py::test_scipy_gridder PASSED                                                                                                                                                                              [ 77%]
../verde/tests/test_scipy.py::test_scipy_gridder_region PASSED                                                                                                                                                                       [ 78%]
../verde/tests/test_scipy.py::test_scipy_gridder_extra_args PASSED                                                                                                                                                                   [ 78%]
../verde/tests/test_scipy.py::test_scipy_gridder_fails PASSED                                                                                                                                                                        [ 79%]
../verde/tests/test_scipy.py::test_scipy_gridder_warns PASSED                                                                                                                                                                        [ 79%]
../verde/tests/test_spline.py::test_spline_cv[serial] PASSED                                                                                                                                                                         [ 80%]
../verde/tests/test_spline.py::test_spline_cv[delayed] FAILED                                                                                                                                                                        [ 80%]
../verde/tests/test_spline.py::test_spline_cv[distributed] PASSED                                                                                                                                                                    [ 81%]
../verde/tests/test_spline.py::test_spline PASSED                                                                                                                                                                                    [ 81%]
../verde/tests/test_spline.py::test_spline_weights PASSED                                                                                                                                                                            [ 82%]
../verde/tests/test_spline.py::test_spline_region PASSED                                                                                                                                                                             [ 82%]
../verde/tests/test_spline.py::test_spline_damping PASSED                                                                                                                                                                            [ 83%]
../verde/tests/test_spline.py::test_spline_warns_weights PASSED                                                                                                                                                                      [ 84%]
../verde/tests/test_spline.py::test_spline_warns_underdetermined PASSED                                                                                                                                                              [ 84%]
../verde/tests/test_spline.py::test_spline_jacobian_implementations PASSED                                                                                                                                                           [ 85%]
../verde/tests/test_spline.py::test_spline_predict_implementations PASSED                                                                                                                                                            [ 85%]
../verde/tests/test_trend.py::test_trend PASSED                                                                                                                                                                                      [ 86%]
../verde/tests/test_trend.py::test_trend_mean PASSED                                                                                                                                                                                 [ 86%]
../verde/tests/test_trend.py::test_trend_weights PASSED                                                                                                                                                                              [ 87%]
../verde/tests/test_trend.py::test_polynomial_combinations_fails PASSED                                                                                                                                                              [ 87%]
../verde/tests/test_trend.py::test_trend_jacobian_fails PASSED                                                                                                                                                                       [ 88%]
../verde/tests/test_utils.py::test_parse_engine PASSED                                                                                                                                                                               [ 89%]
../verde/tests/test_utils.py::test_parse_engine_fails PASSED                                                                                                                                                                         [ 89%]
../verde/tests/test_utils.py::test_dummy_jit PASSED                                                                                                                                                                                  [ 90%]
../verde/tests/test_utils.py::test_kdtree PASSED                                                                                                                                                                                     [ 90%]
../verde/tests/test_utils.py::test_grid_to_table_order PASSED                                                                                                                                                                        [ 91%]
../verde/tests/test_utils.py::test_partition_by_sum_fails_size PASSED                                                                                                                                                                [ 91%]
../verde/tests/test_utils.py::test_partition_by_sum_fails_no_partitions PASSED                                                                                                                                                       [ 92%]
../verde/tests/test_utils.py::test_make_xarray_grid PASSED                                                                                                                                                                           [ 92%]
../verde/tests/test_utils.py::test_make_xarray_grid_multiple_data PASSED                                                                                                                                                             [ 93%]
../verde/tests/test_utils.py::test_make_xarray_grid_extra_coords PASSED                                                                                                                                                              [ 93%]
../verde/tests/test_utils.py::test_make_xarray_grid_invalid_names PASSED                                                                                                                                                             [ 94%]
../verde/tests/test_utils.py::test_make_xarray_grid_invalid_extra_coords PASSED                                                                                                                                                      [ 95%]
../verde/tests/test_vector.py::test_vector2d PASSED                                                                                                                                                                                  [ 95%]
../verde/tests/test_vector.py::test_vector2d_weights PASSED                                                                                                                                                                          [ 96%]
../verde/tests/test_vector.py::test_vector2d_fails PASSED                                                                                                                                                                            [ 96%]
../verde/tests/test_vector.py::test_vector2d_jacobian_implementations PASSED                                                                                                                                                         [ 97%]
../verde/tests/test_vector.py::test_vector2d_predict_implementations PASSED                                                                                                                                                          [ 97%]
../verde/tests/test_vector.py::test_vector_trend PASSED                                                                                                                                                                              [ 98%]
../verde/tests/test_vector.py::test_vector_trend_3d PASSED                                                                                                                                                                           [ 98%]
../verde/tests/test_vector.py::test_vector_trend_fails PASSED                                                                                                                                                                        [ 99%]
../verde/tests/test_vector.py::test_vector_trend_weights PASSED                                                                                                                                                                      [100%]

================================================================================================================= FAILURES =================================================================================================================
_____________________________________________________________________________________________ [doctest] verde.model_selection.cross_val_score ______________________________________________________________________________________________
739     parallel with Dask:
740 
741     >>> cross_validator = ShuffleSplit(n_splits=10, random_state=0)
742     >>> scores_delayed = cross_val_score(
743     ...     model, coords, data, cv=cross_validator, delayed=True
744     ... )
745     >>> # The scores are delayed objects.
746     >>> # To actually run the computations, call dask.compute
747     >>> import dask
748     >>> scores = dask.compute(*scores_delayed)
UNEXPECTED EXCEPTION: KilledWorker('fit_score-b8879bdb-79a9-4090-931b-07125e334b59', <Worker 'inproc://192.168.100.106/19232/3', name: 0, memory: 0, processing: 10>)
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest verde.model_selection.cross_val_score[19]>", line 1, in <module>
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/dask/base.py", line 565, in compute
    results = schedule(dsk, keys, **kwargs)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/client.py", line 2654, in get
    results = self.gather(packed, asynchronous=asynchronous, direct=direct)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/client.py", line 1963, in gather
    return self.sync(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/client.py", line 837, in sync
    return sync(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/utils.py", line 351, in sync
    raise exc.with_traceback(tb)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/utils.py", line 334, in f
    result[0] = yield future
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/gen.py", line 762, in run
    value = future.result()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/client.py", line 1828, in _gather
    raise exception.with_traceback(traceback)
distributed.scheduler.KilledWorker: ('fit_score-b8879bdb-79a9-4090-931b-07125e334b59', <Worker 'inproc://192.168.100.106/19232/3', name: 0, memory: 0, processing: 10>)
/home/santi/git/verde/verde/model_selection.py:748: UnexpectedException
----------------------------------------------------------------------------------------------------------- Captured stderr call -----------------------------------------------------------------------------------------------------------
distributed.core - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.worker - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
tornado.application - ERROR - Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOLoop object at 0x7f0ebad17460>>, <Task finished name='Task-22' coro=<Worker.handle_scheduler() done, defined at /home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py:988> exception=TypeError("'tuple' object does not support item assignment")>)
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 765, in _discard_future_result
    future.result()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.core - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.worker - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
tornado.application - ERROR - Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOLoop object at 0x7f0ebad17460>>, <Task finished name='Task-74' coro=<Worker.handle_scheduler() done, defined at /home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py:988> exception=TypeError("'tuple' object does not support item assignment")>)
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 765, in _discard_future_result
    future.result()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.core - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.worker - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
tornado.application - ERROR - Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOLoop object at 0x7f0ebad17460>>, <Task finished name='Task-89' coro=<Worker.handle_scheduler() done, defined at /home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py:988> exception=TypeError("'tuple' object does not support item assignment")>)
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 765, in _discard_future_result
    future.result()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.core - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.worker - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
tornado.application - ERROR - Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOLoop object at 0x7f0ebad17460>>, <Task finished name='Task-103' coro=<Worker.handle_scheduler() done, defined at /home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py:988> exception=TypeError("'tuple' object does not support item assignment")>)
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 765, in _discard_future_result
    future.result()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
_________________________________________________________________________________________________________ test_spline_cv[delayed] __________________________________________________________________________________________________________

delayed = True, client = None, engine = 'numpy'

    @pytest.mark.parametrize(
        "delayed,client,engine",
        [
            (False, None, "auto"),
            (True, None, "numpy"),
            (False, Client(processes=False), "numpy"),
        ],
        ids=["serial", "delayed", "distributed"],
    )
    def test_spline_cv(delayed, client, engine):
        "See if the cross-validated spline solution matches the synthetic data"
        region = (100, 500, -800, -700)
        synth = CheckerBoard(region=region)
        data = synth.scatter(size=1500, random_state=0)
        coords = (data.easting, data.northing)
        # Can't test on many configurations because it takes too long for regular
        # testing
>       spline = SplineCV(
            dampings=[None, 1e3],
            mindists=[1e-7, 1e6],
            cv=ShuffleSplit(n_splits=1, random_state=0),
            delayed=delayed,
            client=client,
            engine=engine,
        ).fit(coords, data.scalars)

../verde/tests/test_spline.py:34: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../verde/spline.py:225: in fit
    best = best.compute()
../../../.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/dask/base.py:283: in compute
    (result,) = compute(self, traverse=False, **kwargs)
../../../.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/dask/base.py:565: in compute
    results = schedule(dsk, keys, **kwargs)
../../../.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/client.py:2654: in get
    results = self.gather(packed, asynchronous=asynchronous, direct=direct)
../../../.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/client.py:1963: in gather
    return self.sync(
../../../.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/client.py:837: in sync
    return sync(
../../../.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/utils.py:351: in sync
    raise exc.with_traceback(tb)
../../../.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/utils.py:334: in f
    result[0] = yield future
../../../.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/gen.py:762: in run
    value = future.result()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Client: 'inproc://192.168.100.106/19232/1' processes=0 threads=0, memory=0 B>, futures = [[<Future: cancelled, key: argmax-7c6a3b07-a6d2-45d4-a2d1-7b522ba7d25e>]], errors = 'raise', direct = True, local_worker = None

    async def _gather(self, futures, errors="raise", direct=None, local_worker=None):
        unpacked, future_set = unpack_remotedata(futures, byte_keys=True)
        keys = [stringify(future.key) for future in future_set]
        bad_data = dict()
        data = {}

        if direct is None:
            direct = self.direct_to_workers
        if direct is None:
            try:
                w = get_worker()
            except Exception:
                direct = False
            else:
                if w.scheduler.address == self.scheduler.address:
                    direct = True

        async def wait(k):
            """ Want to stop the All(...) early if we find an error """
            st = self.futures[k]
            await st.wait()
            if st.status != "finished" and errors == "raise":
                raise AllExit()

        while True:
            logger.debug("Waiting on futures to clear before gather")

            with suppress(AllExit):
                await All(
                    [wait(key) for key in keys if key in self.futures],
                    quiet_exceptions=AllExit,
                )

            failed = ("error", "cancelled")

            exceptions = set()
            bad_keys = set()
            for key in keys:
                if key not in self.futures or self.futures[key].status in failed:
                    exceptions.add(key)
                    if errors == "raise":
                        try:
                            st = self.futures[key]
                            exception = st.exception
                            traceback = st.traceback
                        except (KeyError, AttributeError):
                            exc = CancelledError(key)
                        else:
>                           raise exception.with_traceback(traceback)
E                           distributed.scheduler.KilledWorker: ('fit_score-1bc940d0-bced-4e42-988f-90c546e01731', <Worker 'inproc://192.168.100.106/19232/3', name: 0, memory: 0, processing: 4>)

../../../.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/client.py:1828: KilledWorker
----------------------------------------------------------------------------------------------------------- Captured stderr call -----------------------------------------------------------------------------------------------------------
distributed.core - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.worker - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
tornado.application - ERROR - Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOLoop object at 0x7f0ebad17460>>, <Task finished name='Task-121' coro=<Worker.handle_scheduler() done, defined at /home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py:988> exception=TypeError("'tuple' object does not support item assignment")>)
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 765, in _discard_future_result
    future.result()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.core - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.worker - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
tornado.application - ERROR - Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOLoop object at 0x7f0ebad17460>>, <Task finished name='Task-325' coro=<Worker.handle_scheduler() done, defined at /home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py:988> exception=TypeError("'tuple' object does not support item assignment")>)
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 765, in _discard_future_result
    future.result()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.core - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.worker - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
tornado.application - ERROR - Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOLoop object at 0x7f0ebad17460>>, <Task finished name='Task-341' coro=<Worker.handle_scheduler() done, defined at /home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py:988> exception=TypeError("'tuple' object does not support item assignment")>)
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 765, in _discard_future_result
    future.result()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.core - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
distributed.worker - ERROR - 'tuple' object does not support item assignment
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
tornado.application - ERROR - Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOLoop object at 0x7f0ebad17460>>, <Task finished name='Task-356' coro=<Worker.handle_scheduler() done, defined at /home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py:988> exception=TypeError("'tuple' object does not support item assignment")>)
Traceback (most recent call last):
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 741, in _run_callback
    ret = callback()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/tornado/ioloop.py", line 765, in _discard_future_result
    future.result()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/worker.py", line 990, in handle_scheduler
    await self.handle_stream(
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/core.py", line 554, in handle_stream
    msgs = await comm.read()
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/comm/inproc.py", line 199, in read
    msg = nested_deserialize(msg)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 609, in nested_deserialize
    return replace_inner(x)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 601, in replace_inner
    x[k] = replace_inner(v)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 594, in replace_inner
    x[k] = deserialize(v.header, v.frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 384, in deserialize
    return loads(header, frames)
  File "/home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/protocol/serialize.py", line 77, in pickle_loads
    buffers[i] = memoryview(bytearray(mv)).cast(mv.format, mv.shape)
TypeError: 'tuple' object does not support item assignment
============================================================================================================= warnings summary =============================================================================================================
../../../.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/numba/core/types/__init__.py:108
  /home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/numba/core/types/__init__.py:108: DeprecationWarning: `np.long` is a deprecated alias for `np.compat.long`. To silence this warning, use `np.compat.long` by itself. In the likely event your code does not need to work on Python 2 you can use the builtin `int` for which `np.compat.long` is itself an alias. Doing this will not modify any behaviour and is safe. When replacing `np.long`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    long_ = _make_signed(np.long)

../../../.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/numba/core/types/__init__.py:109
  /home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/numba/core/types/__init__.py:109: DeprecationWarning: `np.long` is a deprecated alias for `np.compat.long`. To silence this warning, use `np.compat.long` by itself. In the likely event your code does not need to work on Python 2 you can use the builtin `int` for which `np.compat.long` is itself an alias. Doing this will not modify any behaviour and is safe. When replacing `np.long`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    ulong = _make_unsigned(np.long)

verde/coordinates.py::verde.coordinates.inside
verde/coordinates.py::verde.coordinates.inside
verde/coordinates.py::verde.coordinates.inside
  /home/santi/git/verde/verde/coordinates.py:693: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    out = np.empty_like(easting, dtype=np.bool)

verde/coordinates.py: 12 warnings
  /home/santi/git/verde/verde/coordinates.py:694: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
    tmp = tuple(np.empty_like(easting, dtype=np.bool) for i in range(4))

verde/trend.py::verde.trend.Trend.jacobian
  <doctest verde.trend.Trend.jacobian[3]>:1: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

verde/trend.py::verde.trend.Trend.jacobian
  <doctest verde.trend.Trend.jacobian[4]>:1: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

verde/tests/test_datasets.py::test_fetch_rio_magnetic
  /home/santi/git/verde/verde/datasets/sample_data.py:183: DeprecationWarning: The Rio magnetic anomaly dataset is deprecated and will be removed in Verde v2.0.0. Use a different dataset instead.
    warnings.warn(

verde/tests/test_datasets.py::test_setup_rio_magnetic
  /home/santi/git/verde/verde/datasets/sample_data.py:216: DeprecationWarning: The Rio magnetic anomaly dataset is deprecated and will be removed in Verde v2.0.0. Use a different dataset instead.
    warnings.warn(

verde/tests/test_datasets.py::test_setup_cartopy_backward
  /home/santi/git/verde/verde/tests/test_datasets.py:146: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance.  In a future version, a new instance will always be created and returned.  Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.
    ax = plt.subplot(111, projection=ccrs.Mercator())

verde/tests/test_datasets.py::test_setup_cartopy_backward
  /home/santi/git/verde/verde/tests/test_datasets.py:149: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance.  In a future version, a new instance will always be created and returned.  Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.
    ax = plt.subplot(111, projection=ccrs.Mercator())

verde/tests/test_model_selection.py::test_cross_val_score_client
  /home/santi/.anaconda3/envs/verde-3.8/lib/python3.8/site-packages/distributed/node.py:151: UserWarning: Port 8787 is already in use.
  Perhaps you already have a cluster running?
  Hosting the HTTP server on port 40187 instead
    warnings.warn(

verde/tests/test_model_selection.py::test_cross_val_score_client
  /home/santi/git/verde/verde/model_selection.py:757: DeprecationWarning: The 'client' parameter of 'verde.cross_val_score' is deprecated and will be removed in Verde 2.0.0. Use the 'delayed' parameter instead.
    warnings.warn(

verde/tests/test_spline.py::test_spline_cv[distributed]
  /home/santi/git/verde/verde/spline.py:144: DeprecationWarning: The 'client' parameter of 'verde.SplineCV' is deprecated and will be removed in Verde 2.0.0. Use the 'delayed' parameter instead.
    warnings.warn(

verde/tests/test_vector.py::test_vector2d
verde/tests/test_vector.py::test_vector2d_weights
verde/tests/test_vector.py::test_vector2d_fails
verde/tests/test_vector.py::test_vector2d_jacobian_implementations
verde/tests/test_vector.py::test_vector2d_jacobian_implementations
verde/tests/test_vector.py::test_vector2d_predict_implementations
verde/tests/test_vector.py::test_vector2d_predict_implementations
  /home/santi/git/verde/verde/vector.py:233: DeprecationWarning: VectorSpline2D is deprecated and will be removed in Verde v2.0.0. Please use the implementation in the Erizo package instead (https://github.com/fatiando/erizo).
    warnings.warn(

-- Docs: https://docs.pytest.org/en/stable/warnings.html

----------- coverage: platform linux, python 3.8.8-final-0 -----------
Name                                                  Stmts   Miss Branch BrPart  Cover   Missing
-------------------------------------------------------------------------------------------------
/home/santi/git/verde/verde/base/__init__.py              3      0      0      0   100%
/home/santi/git/verde/verde/base/base_classes.py        114      0     40      0   100%
/home/santi/git/verde/verde/base/least_squares.py        14      0      4      0   100%
/home/santi/git/verde/verde/base/utils.py                61      0     44      0   100%
/home/santi/git/verde/verde/blockreduce.py               99      0     63      0   100%
/home/santi/git/verde/verde/chain.py                     30      0     14      0   100%
/home/santi/git/verde/verde/coordinates.py              180      0     92      0   100%
/home/santi/git/verde/verde/datasets/__init__.py          2      0      0      0   100%
/home/santi/git/verde/verde/datasets/sample_data.py      57      2      8      0    97%   14-15
/home/santi/git/verde/verde/datasets/synthetic.py        29      0      4      0   100%
/home/santi/git/verde/verde/distances.py                 12      0      2      0   100%
/home/santi/git/verde/verde/io.py                        37      0     20      0   100%
/home/santi/git/verde/verde/mask.py                      42      0     22      0   100%
/home/santi/git/verde/verde/model_selection.py           95      0     43      0   100%
/home/santi/git/verde/verde/projections.py               38      0     12      0   100%
/home/santi/git/verde/verde/scipygridder.py              30      0      6      0   100%
/home/santi/git/verde/verde/spline.py                   132     13     34      0    87%   18-20, 541-548, 554-559
/home/santi/git/verde/verde/trend.py                     40      0     10      0   100%
/home/santi/git/verde/verde/utils.py                    100      4     40      0    97%   14-15, 19-20
/home/santi/git/verde/verde/vector.py                   125     22     37      1    81%   17-19, 286->288, 449-459, 465-476
/home/santi/git/verde/verde/version.py                    4      0      0      0   100%
-------------------------------------------------------------------------------------------------
TOTAL                                                  1244     41    495      1    97%

========================================================================================================= short test summary info ==========================================================================================================
FAILED ../verde/model_selection.py::verde.model_selection.cross_val_score
FAILED ../verde/tests/test_spline.py::test_spline_cv[delayed] - distributed.scheduler.KilledWorker: ('fit_score-1bc940d0-bced-4e42-988f-90c546e01731', <Worker 'inproc://192.168.100.106/19232/3', name: 0, memory: 0, processing: 4>)
=============================================================================================== 2 failed, 180 passed, 33 warnings in 20.00s ================================================================================================
make: *** [Makefile:26: test] Error 1

System information

output of conda list 

# packages in environment at /home/santi/.anaconda3/envs/verde-3.8:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
alabaster                 0.7.12                     py_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
astroid                   2.3.3                    py38_1    conda-forge
attrs                     20.3.0             pyhd3deb0d_0    conda-forge
babel                     2.9.0              pyhd3deb0d_0    conda-forge
black                     20.8b1                     py_1    conda-forge
bokeh                     2.3.0            py38h578d9bd_0    conda-forge
brotlipy                  0.7.0           py38h497a2fe_1001    conda-forge
c-ares                    1.17.1               h7f98852_1    conda-forge
ca-certificates           2020.12.5            ha878542_0    conda-forge
cartopy                   0.18.0          py38hab71064_13    conda-forge
certifi                   2020.12.5        py38h578d9bd_1    conda-forge
cffi                      1.14.5           py38ha65f79e_0    conda-forge
chardet                   4.0.0            py38h578d9bd_1    conda-forge
click                     7.1.2              pyh9f0ad1d_0    conda-forge
cloudpickle               1.6.0                      py_0    conda-forge
coverage                  5.5              py38h497a2fe_0    conda-forge
cryptography              3.4.6            py38ha5dfef3_0    conda-forge
cycler                    0.10.0                     py_2    conda-forge
cytoolz                   0.11.0           py38h497a2fe_3    conda-forge
dask                      2021.3.0           pyhd8ed1ab_0    conda-forge
dask-core                 2021.3.0           pyhd8ed1ab_0    conda-forge
dataclasses               0.8                pyhc8e2a94_1    conda-forge
dbus                      1.13.6               hfdff14a_1    conda-forge
distributed               2021.3.0         py38h578d9bd_0    conda-forge
docutils                  0.16             py38h578d9bd_3    conda-forge
expat                     2.2.10               h9c3ff4c_0    conda-forge
flake8                    3.8.4                      py_0    conda-forge
fontconfig                2.13.1            hba837de_1004    conda-forge
freetype                  2.10.4               h0708190_1    conda-forge
fsspec                    0.8.7              pyhd8ed1ab_0    conda-forge
geos                      3.9.1                h9c3ff4c_2    conda-forge
gettext                   0.19.8.1          h0b5b191_1005    conda-forge
glib                      2.66.7               h9c3ff4c_1    conda-forge
glib-tools                2.66.7               h9c3ff4c_1    conda-forge
gst-plugins-base          1.18.3               h04508c2_0    conda-forge
gstreamer                 1.18.3               h3560a44_0    conda-forge
heapdict                  1.0.1                      py_0    conda-forge
icu                       68.1                 h58526e2_0    conda-forge
idna                      2.10               pyh9f0ad1d_0    conda-forge
imagesize                 1.2.0                      py_0    conda-forge
importlib-metadata        3.7.2            py38h578d9bd_0    conda-forge
iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
isort                     4.3.21           py38h32f6830_1    conda-forge
jinja2                    2.11.3             pyh44b312d_0    conda-forge
joblib                    1.0.1              pyhd8ed1ab_0    conda-forge
jpeg                      9d                   h36c2ea0_0    conda-forge
kiwisolver                1.3.1            py38h1fd1430_1    conda-forge
krb5                      1.17.2               h926e7f8_0    conda-forge
lazy-object-proxy         1.5.2            py38h497a2fe_1    conda-forge
lcms2                     2.12                 hddcbb42_0    conda-forge
ld_impl_linux-64          2.35.1               hea4e1c9_2    conda-forge
libblas                   3.9.0                8_openblas    conda-forge
libcblas                  3.9.0                8_openblas    conda-forge
libclang                  11.1.0          default_ha53f305_0    conda-forge
libcurl                   7.75.0               hc4aaa36_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               hcdb4288_3    conda-forge
libffi                    3.3                  h58526e2_2    conda-forge
libgcc-ng                 9.3.0               h2828fa1_18    conda-forge
libgfortran-ng            9.3.0               hff62375_18    conda-forge
libgfortran5              9.3.0               hff62375_18    conda-forge
libglib                   2.66.7               h3e27bee_1    conda-forge
libgomp                   9.3.0               h2828fa1_18    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0                8_openblas    conda-forge
libllvm10                 10.0.1               he513fc3_3    conda-forge
libllvm11                 11.1.0               hf817b99_0    conda-forge
libnghttp2                1.43.0               h812cca2_0    conda-forge
libopenblas               0.3.12          pthreads_h4812303_1    conda-forge
libpng                    1.6.37               h21135ba_2    conda-forge
libpq                     13.1                 hfd2b0eb_2    conda-forge
libssh2                   1.9.0                ha56f1ee_6    conda-forge
libstdcxx-ng              9.3.0               h6de172a_18    conda-forge
libtiff                   4.2.0                hdc55705_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libwebp-base              1.2.0                h7f98852_0    conda-forge
libxcb                    1.13              h7f98852_1003    conda-forge
libxkbcommon              1.0.3                he3ba5ed_0    conda-forge
libxml2                   2.9.10               h72842e0_3    conda-forge
llvmlite                  0.35.0           py38h4630a5e_1    conda-forge
locket                    0.2.0                      py_2    conda-forge
lz4-c                     1.9.3                h9c3ff4c_0    conda-forge
markupsafe                1.1.1            py38h497a2fe_3    conda-forge
matplotlib                3.3.4            py38h578d9bd_0    conda-forge
matplotlib-base           3.3.4            py38h0efea84_0    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
more-itertools            8.7.0              pyhd8ed1ab_0    conda-forge
msgpack-python            1.0.2            py38h1fd1430_1    conda-forge
mypy_extensions           0.4.3            py38h578d9bd_3    conda-forge
mysql-common              8.0.23               ha770c72_1    conda-forge
mysql-libs                8.0.23               h935591d_1    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
nspr                      4.29                 h9c3ff4c_1    conda-forge
nss                       3.62                 hb5efdd6_0    conda-forge
numba                     0.52.0           py38h51da96c_0    conda-forge
numpy                     1.20.1           py38h18fd61f_0    conda-forge
numpydoc                  1.1.0                      py_1    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
openssl                   1.1.1j               h7f98852_0    conda-forge
packaging                 20.9               pyh44b312d_0    conda-forge
pandas                    1.2.3            py38h51da96c_0    conda-forge
partd                     1.1.0                      py_0    conda-forge
pathspec                  0.8.1              pyhd3deb0d_0    conda-forge
pcre                      8.44                 he1b5a44_0    conda-forge
pillow                    8.1.2            py38ha0e1e83_0    conda-forge
pip                       21.0.1             pyhd8ed1ab_0    conda-forge
pluggy                    0.13.1           py38h578d9bd_4    conda-forge
pooch                     1.3.0              pyhd8ed1ab_0    conda-forge
proj                      7.2.0                h277dcde_2    conda-forge
psutil                    5.8.0            py38h497a2fe_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
py                        1.10.0             pyhd3deb0d_0    conda-forge
pycodestyle               2.6.0              pyh9f0ad1d_0    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pyflakes                  2.2.0              pyh9f0ad1d_0    conda-forge
pygments                  2.8.1              pyhd8ed1ab_0    conda-forge
pykdtree                  1.3.4            py38h0b5ebd8_0    conda-forge
pylint                    2.4.4                    py38_0    conda-forge
pyopenssl                 20.0.1             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pyproj                    3.0.1            py38h16ecdd7_0    conda-forge
pyqt                      5.12.3           py38h578d9bd_7    conda-forge
pyqt-impl                 5.12.3           py38h7400c14_7    conda-forge
pyqt5-sip                 4.19.18          py38h709712a_7    conda-forge
pyqtchart                 5.12             py38h7400c14_7    conda-forge
pyqtwebengine             5.12.1           py38h7400c14_7    conda-forge
pyshp                     2.1.3              pyh44b312d_0    conda-forge
pysocks                   1.7.1            py38h578d9bd_3    conda-forge
pytest                    6.2.2            py38h578d9bd_0    conda-forge
pytest-cov                2.11.1             pyh44b312d_0    conda-forge
pytest-mpl                0.12               pyhd3deb0d_0    conda-forge
python                    3.8.8           hffdb5ce_0_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.8                      1_cp38    conda-forge
pytz                      2021.1             pyhd8ed1ab_0    conda-forge
pyyaml                    5.4.1            py38h497a2fe_0    conda-forge
qt                        5.12.9               hda022c4_4    conda-forge
readline                  8.0                  he28a2e2_2    conda-forge
regex                     2020.11.13       py38h497a2fe_1    conda-forge
requests                  2.25.1             pyhd3deb0d_0    conda-forge
scikit-learn              0.24.1           py38h658cfdd_0    conda-forge
scipy                     1.6.0            py38hb2138dd_0    conda-forge
setuptools                49.6.0           py38h578d9bd_3    conda-forge
shapely                   1.7.1            py38h4fc1155_4    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
snowballstemmer           2.1.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.3.0              pyhd8ed1ab_0    conda-forge
sphinx                    2.2.1                      py_0    conda-forge
sphinx-gallery            0.8.2              pyhd8ed1ab_0    conda-forge
sphinx_rtd_theme          0.4.3                      py_0    conda-forge
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    1.0.3                      py_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.4                      py_0    conda-forge
sqlite                    3.34.0               h74cdb3f_0    conda-forge
tblib                     1.6.0                      py_0    conda-forge
threadpoolctl             2.1.0              pyh5ca1d4c_0    conda-forge
tk                        8.6.10               h21135ba_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
toolz                     0.11.1                     py_0    conda-forge
tornado                   6.1              py38h497a2fe_1    conda-forge
typed-ast                 1.4.2            py38h497a2fe_0    conda-forge
typing_extensions         3.7.4.3                    py_0    conda-forge
urllib3                   1.26.3             pyhd8ed1ab_0    conda-forge
verde                     1.5.0+16.g0a98d6d.dirty           dev_0    
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
wrapt                     1.12.1           py38h497a2fe_3    conda-forge
xarray                    0.17.0             pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h516909a_0    conda-forge
zict                      2.0.0                      py_0    conda-forge
zipp                      3.4.1              pyhd8ed1ab_0    conda-forge
zlib                      1.2.11            h516909a_1010    conda-forge
zstd                      1.4.9                ha95c52a_0    conda-forge
santisoler commented 3 years ago

Tests pass on Python 3.8 and Dask 2.30.0

output of conda list 
# packages in environment at /home/santi/.anaconda3/envs/verde-3.8:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
alabaster                 0.7.12                     py_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
astroid                   2.3.3                    py38_1    conda-forge
attrs                     20.3.0             pyhd3deb0d_0    conda-forge
babel                     2.9.0              pyhd3deb0d_0    conda-forge
black                     20.8b1                     py_1    conda-forge
bokeh                     2.3.0            py38h578d9bd_0    conda-forge
brotlipy                  0.7.0           py38h497a2fe_1001    conda-forge
c-ares                    1.17.1               h7f98852_1    conda-forge
ca-certificates           2020.12.5            ha878542_0    conda-forge
cartopy                   0.18.0          py38hab71064_13    conda-forge
certifi                   2020.12.5        py38h578d9bd_1    conda-forge
cffi                      1.14.5           py38ha65f79e_0    conda-forge
chardet                   4.0.0            py38h578d9bd_1    conda-forge
click                     7.1.2              pyh9f0ad1d_0    conda-forge
cloudpickle               1.6.0                      py_0    conda-forge
coverage                  5.5              py38h497a2fe_0    conda-forge
cryptography              3.4.6            py38ha5dfef3_0    conda-forge
cycler                    0.10.0                     py_2    conda-forge
cytoolz                   0.11.0           py38h497a2fe_3    conda-forge
dask                      2.30.0                     py_0    conda-forge
dask-core                 2.30.0                     py_0    conda-forge
dataclasses               0.8                pyhc8e2a94_1    conda-forge
dbus                      1.13.6               hfdff14a_1    conda-forge
distributed               2.30.1           py38h578d9bd_0    conda-forge
docutils                  0.16             py38h578d9bd_3    conda-forge
expat                     2.2.10               h9c3ff4c_0    conda-forge
flake8                    3.8.4                      py_0    conda-forge
fontconfig                2.13.1            hba837de_1004    conda-forge
freetype                  2.10.4               h0708190_1    conda-forge
fsspec                    0.8.7              pyhd8ed1ab_0    conda-forge
geos                      3.9.1                h9c3ff4c_2    conda-forge
gettext                   0.19.8.1          h0b5b191_1005    conda-forge
glib                      2.66.7               h9c3ff4c_1    conda-forge
glib-tools                2.66.7               h9c3ff4c_1    conda-forge
gst-plugins-base          1.18.3               h04508c2_0    conda-forge
gstreamer                 1.18.3               h3560a44_0    conda-forge
heapdict                  1.0.1                      py_0    conda-forge
icu                       68.1                 h58526e2_0    conda-forge
idna                      2.10               pyh9f0ad1d_0    conda-forge
imagesize                 1.2.0                      py_0    conda-forge
importlib-metadata        3.7.2            py38h578d9bd_0    conda-forge
iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
isort                     4.3.21           py38h32f6830_1    conda-forge
jinja2                    2.11.3             pyh44b312d_0    conda-forge
joblib                    1.0.1              pyhd8ed1ab_0    conda-forge
jpeg                      9d                   h36c2ea0_0    conda-forge
kiwisolver                1.3.1            py38h1fd1430_1    conda-forge
krb5                      1.17.2               h926e7f8_0    conda-forge
lazy-object-proxy         1.5.2            py38h497a2fe_1    conda-forge
lcms2                     2.12                 hddcbb42_0    conda-forge
ld_impl_linux-64          2.35.1               hea4e1c9_2    conda-forge
libblas                   3.9.0                8_openblas    conda-forge
libcblas                  3.9.0                8_openblas    conda-forge
libclang                  11.1.0          default_ha53f305_0    conda-forge
libcurl                   7.75.0               hc4aaa36_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               hcdb4288_3    conda-forge
libffi                    3.3                  h58526e2_2    conda-forge
libgcc-ng                 9.3.0               h2828fa1_18    conda-forge
libgfortran-ng            9.3.0               hff62375_18    conda-forge
libgfortran5              9.3.0               hff62375_18    conda-forge
libglib                   2.66.7               h3e27bee_1    conda-forge
libgomp                   9.3.0               h2828fa1_18    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0                8_openblas    conda-forge
libllvm10                 10.0.1               he513fc3_3    conda-forge
libllvm11                 11.1.0               hf817b99_0    conda-forge
libnghttp2                1.43.0               h812cca2_0    conda-forge
libopenblas               0.3.12          pthreads_h4812303_1    conda-forge
libpng                    1.6.37               h21135ba_2    conda-forge
libpq                     13.1                 hfd2b0eb_2    conda-forge
libssh2                   1.9.0                ha56f1ee_6    conda-forge
libstdcxx-ng              9.3.0               h6de172a_18    conda-forge
libtiff                   4.2.0                hdc55705_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libwebp-base              1.2.0                h7f98852_0    conda-forge
libxcb                    1.13              h7f98852_1003    conda-forge
libxkbcommon              1.0.3                he3ba5ed_0    conda-forge
libxml2                   2.9.10               h72842e0_3    conda-forge
llvmlite                  0.35.0           py38h4630a5e_1    conda-forge
locket                    0.2.0                      py_2    conda-forge
lz4-c                     1.9.3                h9c3ff4c_0    conda-forge
markupsafe                1.1.1            py38h497a2fe_3    conda-forge
matplotlib                3.3.4            py38h578d9bd_0    conda-forge
matplotlib-base           3.3.4            py38h0efea84_0    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
more-itertools            8.7.0              pyhd8ed1ab_0    conda-forge
msgpack-python            1.0.2            py38h1fd1430_1    conda-forge
mypy_extensions           0.4.3            py38h578d9bd_3    conda-forge
mysql-common              8.0.23               ha770c72_1    conda-forge
mysql-libs                8.0.23               h935591d_1    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
nspr                      4.29                 h9c3ff4c_1    conda-forge
nss                       3.62                 hb5efdd6_0    conda-forge
numba                     0.52.0           py38h51da96c_0    conda-forge
numpy                     1.20.1           py38h18fd61f_0    conda-forge
numpydoc                  1.1.0                      py_1    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
openssl                   1.1.1j               h7f98852_0    conda-forge
packaging                 20.9               pyh44b312d_0    conda-forge
pandas                    1.2.3            py38h51da96c_0    conda-forge
partd                     1.1.0                      py_0    conda-forge
pathspec                  0.8.1              pyhd3deb0d_0    conda-forge
pcre                      8.44                 he1b5a44_0    conda-forge
pillow                    8.1.2            py38ha0e1e83_0    conda-forge
pip                       21.0.1             pyhd8ed1ab_0    conda-forge
pluggy                    0.13.1           py38h578d9bd_4    conda-forge
pooch                     1.3.0              pyhd8ed1ab_0    conda-forge
proj                      7.2.0                h277dcde_2    conda-forge
psutil                    5.8.0            py38h497a2fe_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
py                        1.10.0             pyhd3deb0d_0    conda-forge
pycodestyle               2.6.0              pyh9f0ad1d_0    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pyflakes                  2.2.0              pyh9f0ad1d_0    conda-forge
pygments                  2.8.1              pyhd8ed1ab_0    conda-forge
pykdtree                  1.3.4            py38h0b5ebd8_0    conda-forge
pylint                    2.4.4                    py38_0    conda-forge
pyopenssl                 20.0.1             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pyproj                    3.0.1            py38h16ecdd7_0    conda-forge
pyqt                      5.12.3           py38h578d9bd_7    conda-forge
pyqt-impl                 5.12.3           py38h7400c14_7    conda-forge
pyqt5-sip                 4.19.18          py38h709712a_7    conda-forge
pyqtchart                 5.12             py38h7400c14_7    conda-forge
pyqtwebengine             5.12.1           py38h7400c14_7    conda-forge
pyshp                     2.1.3              pyh44b312d_0    conda-forge
pysocks                   1.7.1            py38h578d9bd_3    conda-forge
pytest                    6.2.2            py38h578d9bd_0    conda-forge
pytest-cov                2.11.1             pyh44b312d_0    conda-forge
pytest-mpl                0.12               pyhd3deb0d_0    conda-forge
python                    3.8.8           hffdb5ce_0_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.8                      1_cp38    conda-forge
pytz                      2021.1             pyhd8ed1ab_0    conda-forge
pyyaml                    5.4.1            py38h497a2fe_0    conda-forge
qt                        5.12.9               hda022c4_4    conda-forge
readline                  8.0                  he28a2e2_2    conda-forge
regex                     2020.11.13       py38h497a2fe_1    conda-forge
requests                  2.25.1             pyhd3deb0d_0    conda-forge
scikit-learn              0.24.1           py38h658cfdd_0    conda-forge
scipy                     1.6.0            py38hb2138dd_0    conda-forge
setuptools                49.6.0           py38h578d9bd_3    conda-forge
shapely                   1.7.1            py38h4fc1155_4    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
snowballstemmer           2.1.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.3.0              pyhd8ed1ab_0    conda-forge
sphinx                    2.2.1                      py_0    conda-forge
sphinx-gallery            0.8.2              pyhd8ed1ab_0    conda-forge
sphinx_rtd_theme          0.4.3                      py_0    conda-forge
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    1.0.3                      py_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.4                      py_0    conda-forge
sqlite                    3.34.0               h74cdb3f_0    conda-forge
tblib                     1.6.0                      py_0    conda-forge
threadpoolctl             2.1.0              pyh5ca1d4c_0    conda-forge
tk                        8.6.10               h21135ba_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
toolz                     0.11.1                     py_0    conda-forge
tornado                   6.1              py38h497a2fe_1    conda-forge
typed-ast                 1.4.2            py38h497a2fe_0    conda-forge
typing_extensions         3.7.4.3                    py_0    conda-forge
urllib3                   1.26.3             pyhd8ed1ab_0    conda-forge
verde                     1.5.0+16.g0a98d6d.dirty           dev_0    
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
wrapt                     1.12.1           py38h497a2fe_3    conda-forge
xarray                    0.17.0             pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h516909a_0    conda-forge
zict                      2.0.0                      py_0    conda-forge
zipp                      3.4.1              pyhd8ed1ab_0    conda-forge
zlib                      1.2.11            h516909a_1010    conda-forge
zstd                      1.4.9                ha95c52a_0    conda-forge
santisoler commented 3 years ago

After manually running the cross_val_score example and the test_spline_cv with the delayed parametrization (delayed = True, client = None, engine = 'numpy') without any error, it seems that the problem is only produced when the example and the test is run through pytest.

cross_val_score example:

import dask
from sklearn.model_selection import ShuffleSplit
import verde as vd

coords = vd.grid_coordinates((0, 10, -10, -5), spacing=0.1)
data = 10 - coords[0] + 0.5 * coords[1]
model = vd.Trend(degree=1)

cross_validator = ShuffleSplit(n_splits=10, random_state=0)
scores_delayed = vd.cross_val_score(
    model, coords, data, cv=cross_validator, delayed=True
)
scores = dask.compute(*scores_delayed)
print(", ".join(["{:.2f}".format(score) for score in scores]))

test_spline_cv:

import verde as vd
from sklearn.model_selection import ShuffleSplit
from dask.distributed import Client

region = (100, 500, -800, -700)
synth = vd.datasets.CheckerBoard(region=region)
data = synth.scatter(size=1500, random_state=0)
coords = (data.easting, data.northing)

delayed, client, engine = True, None, "numpy"

spline = vd.SplineCV(
    dampings=[None, 1e3],
    mindists=[1e-7, 1e6],
    cv=ShuffleSplit(n_splits=1, random_state=0),
    delayed=delayed,
    client=client,
    engine=engine,
).fit(coords, data.scalars)
santisoler commented 3 years ago

This issue might be related to https://github.com/dask/distributed/issues/4570

santisoler commented 3 years ago

Tests pass on Python 3.8 and Dask 2021.2.0

output of conda list 
# packages in environment at /home/santi/.anaconda3/envs/verde-3.8:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
alabaster                 0.7.12                     py_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
astroid                   2.3.3                    py38_1    conda-forge
attrs                     20.3.0             pyhd3deb0d_0    conda-forge
babel                     2.9.0              pyhd3deb0d_0    conda-forge
black                     20.8b1                     py_1    conda-forge
bokeh                     2.3.0            py38h578d9bd_0    conda-forge
brotlipy                  0.7.0           py38h497a2fe_1001    conda-forge
c-ares                    1.17.1               h7f98852_1    conda-forge
ca-certificates           2020.12.5            ha878542_0    conda-forge
cartopy                   0.18.0          py38hab71064_13    conda-forge
certifi                   2020.12.5        py38h578d9bd_1    conda-forge
cffi                      1.14.5           py38ha65f79e_0    conda-forge
chardet                   4.0.0            py38h578d9bd_1    conda-forge
click                     7.1.2              pyh9f0ad1d_0    conda-forge
cloudpickle               1.6.0                      py_0    conda-forge
coverage                  5.5              py38h497a2fe_0    conda-forge
cryptography              3.4.6            py38ha5dfef3_0    conda-forge
cycler                    0.10.0                     py_2    conda-forge
cytoolz                   0.11.0           py38h497a2fe_3    conda-forge
dask                      2021.2.0           pyhd8ed1ab_0    conda-forge
dask-core                 2021.2.0           pyhd8ed1ab_0    conda-forge
dataclasses               0.8                pyhc8e2a94_1    conda-forge
dbus                      1.13.6               hfdff14a_1    conda-forge
distributed               2021.2.0         py38h578d9bd_0    conda-forge
docutils                  0.16             py38h578d9bd_3    conda-forge
expat                     2.2.10               h9c3ff4c_0    conda-forge
flake8                    3.8.4                      py_0    conda-forge
fontconfig                2.13.1            hba837de_1004    conda-forge
freetype                  2.10.4               h0708190_1    conda-forge
fsspec                    0.8.7              pyhd8ed1ab_0    conda-forge
geos                      3.9.1                h9c3ff4c_2    conda-forge
gettext                   0.19.8.1          h0b5b191_1005    conda-forge
glib                      2.66.7               h9c3ff4c_1    conda-forge
glib-tools                2.66.7               h9c3ff4c_1    conda-forge
gst-plugins-base          1.18.3               h04508c2_0    conda-forge
gstreamer                 1.18.3               h3560a44_0    conda-forge
heapdict                  1.0.1                      py_0    conda-forge
icu                       68.1                 h58526e2_0    conda-forge
idna                      2.10               pyh9f0ad1d_0    conda-forge
imagesize                 1.2.0                      py_0    conda-forge
importlib-metadata        3.7.2            py38h578d9bd_0    conda-forge
iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
isort                     4.3.21           py38h32f6830_1    conda-forge
jinja2                    2.11.3             pyh44b312d_0    conda-forge
joblib                    1.0.1              pyhd8ed1ab_0    conda-forge
jpeg                      9d                   h36c2ea0_0    conda-forge
kiwisolver                1.3.1            py38h1fd1430_1    conda-forge
krb5                      1.17.2               h926e7f8_0    conda-forge
lazy-object-proxy         1.5.2            py38h497a2fe_1    conda-forge
lcms2                     2.12                 hddcbb42_0    conda-forge
ld_impl_linux-64          2.35.1               hea4e1c9_2    conda-forge
libblas                   3.9.0                8_openblas    conda-forge
libcblas                  3.9.0                8_openblas    conda-forge
libclang                  11.1.0          default_ha53f305_0    conda-forge
libcurl                   7.75.0               hc4aaa36_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               hcdb4288_3    conda-forge
libffi                    3.3                  h58526e2_2    conda-forge
libgcc-ng                 9.3.0               h2828fa1_18    conda-forge
libgfortran-ng            9.3.0               hff62375_18    conda-forge
libgfortran5              9.3.0               hff62375_18    conda-forge
libglib                   2.66.7               h3e27bee_1    conda-forge
libgomp                   9.3.0               h2828fa1_18    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
liblapack                 3.9.0                8_openblas    conda-forge
libllvm10                 10.0.1               he513fc3_3    conda-forge
libllvm11                 11.1.0               hf817b99_0    conda-forge
libnghttp2                1.43.0               h812cca2_0    conda-forge
libopenblas               0.3.12          pthreads_h4812303_1    conda-forge
libpng                    1.6.37               h21135ba_2    conda-forge
libpq                     13.1                 hfd2b0eb_2    conda-forge
libssh2                   1.9.0                ha56f1ee_6    conda-forge
libstdcxx-ng              9.3.0               h6de172a_18    conda-forge
libtiff                   4.2.0                hdc55705_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libwebp-base              1.2.0                h7f98852_0    conda-forge
libxcb                    1.13              h7f98852_1003    conda-forge
libxkbcommon              1.0.3                he3ba5ed_0    conda-forge
libxml2                   2.9.10               h72842e0_3    conda-forge
llvmlite                  0.35.0           py38h4630a5e_1    conda-forge
locket                    0.2.0                      py_2    conda-forge
lz4-c                     1.9.3                h9c3ff4c_0    conda-forge
markupsafe                1.1.1            py38h497a2fe_3    conda-forge
matplotlib                3.3.4            py38h578d9bd_0    conda-forge
matplotlib-base           3.3.4            py38h0efea84_0    conda-forge
mccabe                    0.6.1                      py_1    conda-forge
more-itertools            8.7.0              pyhd8ed1ab_0    conda-forge
msgpack-python            1.0.2            py38h1fd1430_1    conda-forge
mypy_extensions           0.4.3            py38h578d9bd_3    conda-forge
mysql-common              8.0.23               ha770c72_1    conda-forge
mysql-libs                8.0.23               h935591d_1    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
nspr                      4.29                 h9c3ff4c_1    conda-forge
nss                       3.62                 hb5efdd6_0    conda-forge
numba                     0.52.0           py38h51da96c_0    conda-forge
numpy                     1.20.1           py38h18fd61f_0    conda-forge
numpydoc                  1.1.0                      py_1    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
openssl                   1.1.1j               h7f98852_0    conda-forge
packaging                 20.9               pyh44b312d_0    conda-forge
pandas                    1.2.3            py38h51da96c_0    conda-forge
partd                     1.1.0                      py_0    conda-forge
pathspec                  0.8.1              pyhd3deb0d_0    conda-forge
pcre                      8.44                 he1b5a44_0    conda-forge
pillow                    8.1.2            py38ha0e1e83_0    conda-forge
pip                       21.0.1             pyhd8ed1ab_0    conda-forge
pluggy                    0.13.1           py38h578d9bd_4    conda-forge
pooch                     1.3.0              pyhd8ed1ab_0    conda-forge
proj                      7.2.0                h277dcde_2    conda-forge
psutil                    5.8.0            py38h497a2fe_1    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
py                        1.10.0             pyhd3deb0d_0    conda-forge
pycodestyle               2.6.0              pyh9f0ad1d_0    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pyflakes                  2.2.0              pyh9f0ad1d_0    conda-forge
pygments                  2.8.1              pyhd8ed1ab_0    conda-forge
pykdtree                  1.3.4            py38h0b5ebd8_0    conda-forge
pylint                    2.4.4                    py38_0    conda-forge
pyopenssl                 20.0.1             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pyproj                    3.0.1            py38h16ecdd7_0    conda-forge
pyqt                      5.12.3           py38h578d9bd_7    conda-forge
pyqt-impl                 5.12.3           py38h7400c14_7    conda-forge
pyqt5-sip                 4.19.18          py38h709712a_7    conda-forge
pyqtchart                 5.12             py38h7400c14_7    conda-forge
pyqtwebengine             5.12.1           py38h7400c14_7    conda-forge
pyshp                     2.1.3              pyh44b312d_0    conda-forge
pysocks                   1.7.1            py38h578d9bd_3    conda-forge
pytest                    6.2.2            py38h578d9bd_0    conda-forge
pytest-cov                2.11.1             pyh44b312d_0    conda-forge
pytest-mpl                0.12               pyhd3deb0d_0    conda-forge
python                    3.8.8           hffdb5ce_0_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.8                      1_cp38    conda-forge
pytz                      2021.1             pyhd8ed1ab_0    conda-forge
pyyaml                    5.4.1            py38h497a2fe_0    conda-forge
qt                        5.12.9               hda022c4_4    conda-forge
readline                  8.0                  he28a2e2_2    conda-forge
regex                     2020.11.13       py38h497a2fe_1    conda-forge
requests                  2.25.1             pyhd3deb0d_0    conda-forge
scikit-learn              0.24.1           py38h658cfdd_0    conda-forge
scipy                     1.6.0            py38hb2138dd_0    conda-forge
setuptools                49.6.0           py38h578d9bd_3    conda-forge
shapely                   1.7.1            py38h4fc1155_4    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
snowballstemmer           2.1.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.3.0              pyhd8ed1ab_0    conda-forge
sphinx                    2.2.1                      py_0    conda-forge
sphinx-gallery            0.8.2              pyhd8ed1ab_0    conda-forge
sphinx_rtd_theme          0.4.3                      py_0    conda-forge
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    1.0.3                      py_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.4                      py_0    conda-forge
sqlite                    3.34.0               h74cdb3f_0    conda-forge
tblib                     1.6.0                      py_0    conda-forge
threadpoolctl             2.1.0              pyh5ca1d4c_0    conda-forge
tk                        8.6.10               h21135ba_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
toolz                     0.11.1                     py_0    conda-forge
tornado                   6.1              py38h497a2fe_1    conda-forge
typed-ast                 1.4.2            py38h497a2fe_0    conda-forge
typing_extensions         3.7.4.3                    py_0    conda-forge
urllib3                   1.26.3             pyhd8ed1ab_0    conda-forge
verde                     1.5.0+16.g0a98d6d.dirty           dev_0    
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
wrapt                     1.12.1           py38h497a2fe_3    conda-forge
xarray                    0.17.0             pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h516909a_0    conda-forge
zict                      2.0.0                      py_0    conda-forge
zipp                      3.4.1              pyhd8ed1ab_0    conda-forge
zlib                      1.2.11            h516909a_1010    conda-forge
zstd                      1.4.9                ha95c52a_0    conda-forge