Module compiled against API version 0x10 but this version of numpy is 0xf

[RobBlackwell]

Describe the bug

The example notebook referenced from the getting started documentation no longer seems to work, error is module compiled against API version 0x10 but this version of numpy is 0xf.

I suppose that all the models need to be recompiled so that dependent library versions remain in step?

How To Reproduce

Load the example notebook and run.

It fails with a RuntimeError when you get to resnet_model = load_pretrained_model(model_repo, model='resnet18', allow_install=True)

Environment (generate for Linux, macOS and 64-bit Windows)

Example binder.

Additional information

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xf
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Input In [4], in <cell line: 1>()
----> 1 resnet_model = load_pretrained_model(model_repo, model='resnet18', allow_install=True)
      2 densenet_model = load_pretrained_model(model_repo, model='densenet121')

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scivision/koala.py:57, in koala.<locals>.wrapped_fn(*args, **kwargs)
     55 # now call the function, but log any exceptions
     56 try:
---> 57     output = fn(*args, **kwargs)
     58 except Exception as e:
     59     logger.error(e)

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scivision/io/reader.py:175, in load_pretrained_model(path, branch, allow_install, model, load_multiple, *args, **kwargs)
    172     # try to install the package if necessary
    173     install_package(config, allow_install=allow_install, branch=branch)
--> 175     loaded_models.append(PretrainedModel(config))
    176 if load_multiple:
    177     return loaded_models

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scivision/io/wrapper.py:14, in PretrainedModel.__init__(self, config)
     12 def __init__(self, config: dict):
     13     self._config = config
---> 14     self._plumbing = AutoPlumber(config)

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scivision/io/autoplumber.py:33, in AutoPlumber.__init__(self, config)
     30 def __init__(self, config: dict):
     31 
     32     # import the module and get the model object
---> 33     self._module = importlib.import_module(config["import"])
     34     model = getattr(self._module, config["model"])
     36     # we could instantiate the model using the args here
     37     # args = config["args"]

File /srv/conda/envs/notebook/lib/python3.9/importlib/__init__.py:127, in import_module(name, package)
    125             break
    126         level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)

File <frozen importlib._bootstrap>:1030, in _gcd_import(name, package, level)

File <frozen importlib._bootstrap>:1007, in _find_and_load(name, import_)

File <frozen importlib._bootstrap>:986, in _find_and_load_unlocked(name, import_)

File <frozen importlib._bootstrap>:680, in _load_unlocked(spec)

File <frozen importlib._bootstrap_external>:850, in exec_module(self, module)

File <frozen importlib._bootstrap>:228, in _call_with_frames_removed(f, *args, **kwds)

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scivision_classifier/__init__.py:1, in <module>
----> 1 from .model import vgg16
      2 from .model import vgg19
      3 from .model import resnet18

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scivision_classifier/model.py:4, in <module>
      2 from classification_models.tfkeras import Classifiers
      3 from skimage.filters import gaussian
----> 4 from skimage.transform import resize
      5 from tensorflow.keras.applications.imagenet_utils import decode_predictions
      8 def tidy_predict(self, image: np.ndarray) -> str:

File /srv/conda/envs/notebook/lib/python3.9/site-packages/skimage/transform/__init__.py:4, in <module>
      1 from .hough_transform import (hough_line, hough_line_peaks,
      2                               probabilistic_hough_line, hough_circle,
      3                               hough_circle_peaks, hough_ellipse)
----> 4 from .radon_transform import (radon, iradon, iradon_sart,
      5                               order_angles_golden_ratio)
      6 from .finite_radon_transform import frt2, ifrt2
      7 from .integral import integral_image, integrate

File /srv/conda/envs/notebook/lib/python3.9/site-packages/skimage/transform/radon_transform.py:6, in <module>
      4 from scipy.constants import golden_ratio
      5 from scipy.fft import fft, ifft, fftfreq, fftshift
----> 6 from ._warps import warp
      7 from ._radon_transform import sart_projection_update
      8 from .._shared.utils import convert_to_float

File /srv/conda/envs/notebook/lib/python3.9/site-packages/skimage/transform/_warps.py:9, in <module>
      6 from ._geometric import (SimilarityTransform, AffineTransform,
      7                          ProjectiveTransform)
      8 from ._warps_cy import _warp_fast
----> 9 from ..measure import block_reduce
     11 from .._shared.utils import (get_bound_method_class, safe_as_int, warn,
     12                              convert_to_float, _to_ndimage_mode,
     13                              _validate_interpolation_order,
     14                              channel_as_last_axis,
     15                              deprecate_multichannel_kwarg)
     17 HOMOGRAPHY_TRANSFORMS = (
     18     SimilarityTransform,
     19     AffineTransform,
     20     ProjectiveTransform
     21 )

File /srv/conda/envs/notebook/lib/python3.9/site-packages/skimage/measure/__init__.py:6, in <module>
      3 from ._marching_cubes_classic import mesh_surface_area
      4 from ._regionprops import (regionprops, perimeter,
      5                            perimeter_crofton, euler_number, regionprops_table)
----> 6 from ._polygon import approximate_polygon, subdivide_polygon
      7 from .pnpoly import points_in_poly, grid_points_in_poly
      8 from ._moments import (moments, moments_central, moments_coords,
      9                        moments_coords_central, moments_normalized, centroid,
     10                        moments_hu, inertia_tensor, inertia_tensor_eigvals)

File /srv/conda/envs/notebook/lib/python3.9/site-packages/skimage/measure/_polygon.py:2, in <module>
      1 import numpy as np
----> 2 from scipy import signal
      5 def approximate_polygon(coords, tolerance):
      6     """Approximate a polygonal chain with the specified tolerance.
      7 
      8     It is based on the Douglas-Peucker algorithm.
   (...)
     29     .. [1] https://en.wikipedia.org/wiki/Ramer-Douglas-Peucker_algorithm
     30     """

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scipy/__init__.py:211, in __getattr__(name)
    209 def __getattr__(name):
    210     if name in submodules:
--> 211         return _importlib.import_module(f'scipy.{name}')
    212     else:
    213         try:

File /srv/conda/envs/notebook/lib/python3.9/importlib/__init__.py:127, in import_module(name, package)
    125             break
    126         level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scipy/signal/__init__.py:323, in <module>
    314 from ._spline import (  # noqa: F401
    315     cspline2d,
    316     qspline2d,
   (...)
    319     symiirorder2,
    320 )
    322 from ._bsplines import *
--> 323 from ._filter_design import *
    324 from ._fir_filter_design import *
    325 from ._ltisys import *

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scipy/signal/_filter_design.py:16, in <module>
     13 from numpy.polynomial.polynomial import polyval as npp_polyval
     14 from numpy.polynomial.polynomial import polyvalfromroots
---> 16 from scipy import special, optimize, fft as sp_fft
     17 from scipy.special import comb
     18 from scipy._lib._util import float_factorial

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scipy/__init__.py:211, in __getattr__(name)
    209 def __getattr__(name):
    210     if name in submodules:
--> 211         return _importlib.import_module(f'scipy.{name}')
    212     else:
    213         try:

File /srv/conda/envs/notebook/lib/python3.9/importlib/__init__.py:127, in import_module(name, package)
    125             break
    126         level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scipy/optimize/__init__.py:401, in <module>
      1 """
      2 =====================================================
      3 Optimization and root finding (:mod:`scipy.optimize`)
   (...)
    397 
    398 """
    400 from ._optimize import *
--> 401 from ._minimize import *
    402 from ._root import *
    403 from ._root_scalar import *

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scipy/optimize/_minimize.py:26, in <module>
     24 from ._trustregion_krylov import _minimize_trust_krylov
     25 from ._trustregion_exact import _minimize_trustregion_exact
---> 26 from ._trustregion_constr import _minimize_trustregion_constr
     28 # constrained minimization
     29 from ._lbfgsb_py import _minimize_lbfgsb

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scipy/optimize/_trustregion_constr/__init__.py:4, in <module>
      1 """This module contains the equality constrained SQP solver."""
----> 4 from .minimize_trustregion_constr import _minimize_trustregion_constr
      6 __all__ = ['_minimize_trustregion_constr']

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scipy/optimize/_trustregion_constr/minimize_trustregion_constr.py:5, in <module>
      3 from scipy.sparse.linalg import LinearOperator
      4 from .._differentiable_functions import VectorFunction
----> 5 from .._constraints import (
      6     NonlinearConstraint, LinearConstraint, PreparedConstraint, strict_bounds)
      7 from .._hessian_update_strategy import BFGS
      8 from .._optimize import OptimizeResult

File /srv/conda/envs/notebook/lib/python3.9/site-packages/scipy/optimize/_constraints.py:8, in <module>
      6 from ._optimize import OptimizeWarning
      7 from warnings import warn, catch_warnings, simplefilter
----> 8 from numpy.testing import suppress_warnings
      9 from scipy.sparse import issparse
     12 def _arr_to_scalar(x):
     13     # If x is a numpy array, return x.item().  This will
     14     # fail if the array has more than one element.

File /srv/conda/envs/notebook/lib/python3.9/site-packages/numpy/testing/__init__.py:10, in <module>
      1 """Common test support for all numpy test scripts.
      2 
      3 This single module should provide all the common functionality for numpy tests
   (...)
      6 
      7 """
      8 from unittest import TestCase
---> 10 from ._private.utils import *
     11 from ._private.utils import (_assert_valid_refcount, _gen_alignment_data)
     12 from ._private import extbuild, decorators as dec

File /srv/conda/envs/notebook/lib/python3.9/site-packages/numpy/testing/_private/utils.py:23, in <module>
     20 import numpy as np
     21 from numpy.core import(
     22      intp, float32, empty, arange, array_repr, ndarray, isnat, array)
---> 23 import numpy.linalg.lapack_lite
     25 from io import StringIO
     27 __all__ = [
     28         'assert_equal', 'assert_almost_equal', 'assert_approx_equal',
     29         'assert_array_equal', 'assert_array_less', 'assert_string_equal',
   (...)
     38         'assert_no_gc_cycles', 'break_cycles', 'HAS_LAPACK64', 'IS_PYSTON',
     39         ]

ImportError: numpy.core.multiarray failed to import

[edwardchalstrey1]

Thanks @RobBlackwell unfortunately this seems to be a thorny Binder issue with a mismatch in numpy versions causing issues. At the moment, when I run the notebook locally, this doesn't seem to happen. In either case, the conda environment I am using to run the notebook (or gets used to build the binder) installs numpy via it being a dependency of scivision (by default it is numpy-1.23.3 now). So there must be some quirk of how the binder get's built that I'm misunderstanding - I'll try to look into this further.

[edwardchalstrey1]

@acocac resident Binder expert, do you have any thoughts on this?

[edwardchalstrey1]

Ok, for some reason the binder at https://github.com/alan-turing-institute/scivision called "Launch Binder" works - I will replace the button in the readthedocs (its the same notebook)

[edwardchalstrey1]

ah wait no it failed for me there too

[edwardchalstrey1]

I tested clicking the binder link from the readthedocs link in a private browser and it now works...

[acocac]

@RobBlackwell can you try to replicate @edwardchalstrey1 suggested workaround?

[edwardchalstrey1]

One of my colleague who hasn't opened the link before also had the same issue

[edwardchalstrey1]

and now I just tried again with incognito and it doesn't work

[edwardchalstrey1]

I will try and create a fresh binder link and see if that works

[edwardchalstrey1]

ok yes this worked, I will update the links in the docs