TypeError: expected dtype object, got 'numpy.dtype[float64]'

[hassaniqbal209]

Hi,

I am running the Navier-Stokes notebook and getting the following error after 33% filtering steps of LETKF:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-39-7910f4fa0882> in <module>
      1 results_letkf = letkf.filter(
      2     model, observation_sequence, observation_time_indices,
----> 3     num_particle=50, rng=rng, return_particles=False)

~/anaconda3/envs/dapy/lib/python3.7/site-packages/dapy-0.0.0-py3.7-linux-x86_64.egg/dapy/filters/base.py in filter(self, model, observation_sequence, observation_time_indices, num_particle, rng, return_particles)
    133                         predicted_states,
    134                         observation_sequence[observation_index],
--> 135                         time_index,
    136                     )
    137                     for key, statistic_array in statistics.items():

~/anaconda3/envs/dapy/lib/python3.7/site-packages/dapy-0.0.0-py3.7-linux-x86_64.egg/dapy/filters/local.py in _assimilation_update(self, model, rng, previous_states, predicted_states, observation, time_index)
    107         for node_index in range(model.mesh_size):
    108             local_indices, local_weights = self._observation_indices_and_weights(
--> 109                 node_index, model
    110             )
    111             post_states_mesh[:, :, node_index] = self._local_assimilation_update(

~/anaconda3/envs/dapy/lib/python3.7/site-packages/dapy-0.0.0-py3.7-linux-x86_64.egg/dapy/filters/local.py in _observation_indices_and_weights(self, node_index, model)
     79         )
     80         localisation_weights = self.localisation_weighting_func(
---> 81             observation_distances, self.localisation_radius
     82         )
     83         non_zero_localisation_weights = localisation_weights > 0.0

TypeError: expected dtype object, got 'numpy.dtype[float64]'

For LETPF, I get the same error in the very first filtering time step.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-41-6cfaaaafc366> in <module>
      1 results_letpf = letpf.filter(
      2     model, observation_sequence, observation_time_indices,
----> 3     num_particle=50, rng=rng, return_particles=False)

~/anaconda3/envs/dapy/lib/python3.7/site-packages/dapy-0.0.0-py3.7-linux-x86_64.egg/dapy/filters/base.py in filter(self, model, observation_sequence, observation_time_indices, num_particle, rng, return_particles)
    114                 (num_observation_time, num_particle, model.dim_state), np.nan
    115             )
--> 116         self._perform_model_specific_initialization(model, num_particle)
    117         observation_index = 0
    118         states = None

~/anaconda3/envs/dapy/lib/python3.7/site-packages/dapy-0.0.0-py3.7-linux-x86_64.egg/dapy/filters/local.py in _perform_model_specific_initialization(self, model, num_particle)
    485                     self.localisation_radius,
    486                 )
--> 487                 for p in range(self.partition_of_unity.num_patch)
    488             ],
    489             axis=0,

~/anaconda3/envs/dapy/lib/python3.7/site-packages/dapy-0.0.0-py3.7-linux-x86_64.egg/dapy/filters/local.py in <listcomp>(.0)
    485                     self.localisation_radius,
    486                 )
--> 487                 for p in range(self.partition_of_unity.num_patch)
    488             ],
    489             axis=0,

TypeError: expected dtype object, got 'numpy.dtype[float64]'

In the above cases, I have not changed localization radius, number of particles and kept everything the same as in example notebook. Moreover, I tried increasing number of particles and localization radius, and received the same error at the same time steps for both filters respectively. Do you have any suggestions on what might be causing this and how to resolve ? Thank you.

[matt-graham]

Hi @hassaniqbal209, thanks for the bug report.

Can I check what NumPy and Numba versions you are using? The default for the localisation_weighting_func attribute of the LocalEnsembleTransformKalmanFilter which seems to be where the error is getting raised from the (first) traceback is a JIT-compiled Numba function defined in dapy/utils/localisation.py. From a bit of searching, it seems like this specific error may be due to an incompatibility between the installed NumPy and Numba versions (https://stackoverflow.com/q/67016356). Running the notebook locally in a clean Python 3.8 environment with Numba v0.56.4 and Numpy v1.23.5, the cell with the letkf.filter call runs to completion without any errors.

[hassaniqbal209]

Hi @matt-graham ,

Thank you for your guidance. You are correct the error is due to incompatibility of numpy and numpa, and running it in clean python 3.8 worked! The issue is resolved from my end. Thank you so much for your time.

[matt-graham]

@hassaniqbal209 thanks for the quick response. Good to know this resolved the issue. I'll update the setup script / instructions to make try to avoid this problem happening for others.