dsb-2018/01-data.ipynb ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

[truatpasteurdotfr]

Hi,

I am trying to run the firs example notebook, and I am failing at the very first cell...

miniconda installation, creating the environment from your directions.

conda env create -f EmbedSeg_environment.yml
conda activate EmbedSegEnv
python3 -m pip install -e .
python3 -m ipykernel install --sys-prefix  --name EmbedSegEnv --display-name "EmbedSegEnv"

(instead of --user to install it into the virtualenv instead of $HOME/.local)

(EmbedSegEnv) [tru@sillage EmbedSeg]$ pip3 list |grep numpy
numpy                             1.19.4
(EmbedSegEnv) [tru@sillage EmbedSeg]$ pip3 list |grep hdm
hdmedians                         0.14.1

from tqdm import tqdm

from glob import glob

import tifffile

import numpy as np

import os

from EmbedSeg.utils.preprocess_data import extract_data, split_train_val

from EmbedSeg.utils.generate_crops import *

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-1-54e5f42b447e> in <module>
      5 import os
      6 from EmbedSeg.utils.preprocess_data import extract_data, split_train_val
----> 7 from EmbedSeg.utils.generate_crops import *

~/git/github/juglab/EmbedSeg/EmbedSeg/utils/generate_crops.py in <module>
      5 from scipy.ndimage.morphology import binary_fill_holes
      6 from scipy.spatial import distance_matrix
----> 7 import hdmedians as hd
      8 from numba import jit
      9 

/c7/home/tru/miniconda3/envs/EmbedSegEnv/lib/python3.7/site-packages/hdmedians/__init__.py in <module>
      4 
      5 from .medoid import medoid, nanmedoid
----> 6 from .geomedian import geomedian, nangeomedian

hdmedians/geomedian.pyx in init hdmedians.geomedian()

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

[fjug]

Hmmm... seems like the active conda env does either not have all required packages installed and then uses some incompatible one from somewhere else, or for some reason incompatible packages had been installed.

I’d start from scratch with a fresh conda environment and hope it just works.

Or @MLbyML sees this and has muuuuch better advice... 😉

[truatpasteurdotfr]

conda list --explicit > /tmp/EmbedSeg_environment-explicit.list attached EmbedSeg_environment-explicit.list.txt

[truatpasteurdotfr]

pip3 list > /tmp/pip3.list.txt attached pip3.list.txt

[lmanan]

Hello @truatpasteurdotfr Thank you for opening an issue.

I can potentially suggest a quick fix for now.

1. Please comment out the import of hdmedians package here
2. Remove the hdmedians package from EmbedSeg_environment.yml by deleting this line (Currently hdmedians package is only being imported temporarily just for benchmarking different packages for medoid evaluation but is not, at the moment, being used in the EmbedSeg code. The import will be removed from the EmbedSeg_environment.yml file possibly in the future. )
3. Create the environment again from scratch
   • Remove EmbedSegEnv completely using:

     conda remove -n EmbedSegEnv --all

   • Repeat the instructions from Getting Started section in README.md

     conda env create -f EmbedSeg_environment.yml
     conda activate EmbedSegEnv
     python3 -m pip install -e .
     # python3 -m ipykernel install --user --name EmbedSegEnv --display-name "EmbedSegEnv"
     python3 -m ipykernel install --sys-prefix  --name EmbedSegEnv --display-name "EmbedSegEnv"

Let us know if this helps. Thank you for debugging and using the code!

[truatpasteurdotfr]

:P that seems to have fixed the issue, thx.