[BUG] Brainmapper does not perform classication

[Eddymorphling]

Hi folks, I am trying to use the the new brainmapperCLI tool that uses a ResNet model that I generated with cellfinder. Here is the CLI I use to run brainmapper

# folder paths
signal_folder=/home/scratch/BigStitcher_output/Brain_2_724/dataset/tiffs
background_folder=/home/scratch/BigStitcher_output/Brain_2_724/dataset/tiffs
output_folder=/home/scratch/BigStitcher_output/Brain_2_724/dataset/brainreg-test
trained_model=/home/scratch/BigStitcher_output/cellfinder_retrain/4_retrained_model_v1/model.h5

# an example for brainreg. See the user guide for the specific parameters - https://brainglobe.info/documentation/brainglobe-workflows/brainmapper/data-requirements.html

brainmapper -s $signal_folder -b $background_folder -o $output_folder -v 3.9 3.9 3.9 --orientation sal --atlas allen_mouse_100um --start-plane 400 --end-plane 550 --ball-xy-size 2 --ball-z-size 2 --ball-overlap-fraction 0.4 --soma-spread-factor 0 --trained-model $trained_model --batch-size 128

echo "Brainmapper Complete"

Now everytime I run this only the registration step works well but the cell classification fails with the following error

Traceback (most recent call last):
  File "/home/conda/envs/brainglobe_v1/bin/brainmapper", line 8, in <module>
    sys.exit(main())
  File "/home/conda/envs/brainglobe_v1/lib/python3.10/site-packages/brainglobe_workflows/brainmapper/main.py", line 96, in main
    run_all(args, what_to_run, atlas)
  File "/home/conda/envs/brainglobe_v1/lib/python3.10/site-packages/brainglobe_workflows/brainmapper/main.py", line 157, in run_all
    model_weights = prep.prep_classification(
AttributeError: module 'cellfinder.core.tools.prep' has no attribute 'prep_classification'

If I run the same ResNet model using cellfinder in napari, everything works out well but not using the brainmapper CLI. Any clue what could be the issue here? Thank you.

[adamltyson]

Hi @Eddymorphling I think you may have an old version of brainmapper (sorry, things are moving fairly quickly at the moment). Could you try pip install brainglobe-workflows brainglobe-utils -U?

[Eddymorphling]

Hi @adamltyson I did try that and it seems that these two packages are up to date. Here is my pip list just in case. Also the braingmapper CLI shows me the same error when I use the inbuilt cellfinder model. I tried this on a fresh conda env.

Package                       Version
----------------------------- ------------
absl-py                       2.0.0
alabaster                     0.7.16
app-model                     0.2.4
appdirs                       1.4.4
asttokens                     2.4.1
astunparse                    1.6.3
attrs                         23.2.0
Babel                         2.14.0
beautifulsoup4                4.12.2
bg-atlasapi                   1.0.2
bg_space                      0.6.0
blessed                       1.20.0
blosc2                        2.4.0
brainglobe                    1.0.0
brainglobe-heatmap            0.5.2
brainglobe-napari-io          0.3.2
brainglobe-segmentation       1.2.0
brainglobe-utils              0.3.4
brainglobe-workflows          1.1.0
brainreg                      1.0.3
brainrender                   2.1.3
brainrender-napari            0.0.2
bs4                           0.0.1
build                         1.0.3
cachetools                    5.3.2
cachey                        0.2.1
cellfinder                    1.1.0
certifi                       2023.11.17
charset-normalizer            3.3.2
click                         8.1.7
cloudpickle                   3.0.0
comm                          0.2.1
configobj                     5.0.8
contourpy                     1.2.0
cycler                        0.12.1
dask                          2023.12.1
debugpy                       1.8.0
decorator                     5.1.1
Deprecated                    1.2.14
docstring-parser              0.15
docutils                      0.17.1
exceptiongroup                1.2.0
executing                     2.0.1
fancylog                      0.3.0
flatbuffers                   23.5.26
fonttools                     4.47.2
freetype-py                   2.4.0
fsspec                        2023.12.2
gast                          0.4.0
google                        3.0.0
google-auth                   2.26.2
google-auth-oauthlib          0.4.6
google-pasta                  0.2.0
grpcio                        1.60.0
h5py                          3.10.0
HeapDict                      1.0.1
hsluv                         5.0.4
idna                          3.6
imagecodecs                   2024.1.1
imageio                       2.33.1
imagesize                     1.4.1
imio                          0.3.0
importlib-metadata            7.0.1
in-n-out                      0.1.9
ipykernel                     6.28.0
ipython                       8.20.0
ipywidgets                    8.1.1
jedi                          0.19.1
Jinja2                        3.1.3
joblib                        1.3.2
jsonschema                    4.20.0
jsonschema-specifications     2023.12.1
jupyter_client                8.6.0
jupyter_core                  5.7.1
jupyterlab-widgets            3.0.9
k3d                           2.16.1
keras                         2.11.0
kiwisolver                    1.4.5
lazy_loader                   0.3
libclang                      16.0.6
llvmlite                      0.41.1
locket                        1.0.0
loguru                        0.7.2
magicgui                      0.8.1
Markdown                      3.5.2
markdown-it-py                3.0.0
MarkupSafe                    2.1.3
matplotlib                    3.8.2
matplotlib-inline             0.1.6
mdurl                         0.1.2
meshio                        5.3.4
morphapi                      0.2.1
MorphIO                       3.3.7
msgpack                       1.0.7
multiprocessing-logging       0.3.4
mypy-extensions               1.0.0
myterial                      1.2.1
napari                        0.4.18
napari-console                0.0.9
napari-ndtiffs                0.2.1
napari-plugin-engine          0.2.0
napari-plugin-manager         0.1.0a2
napari-svg                    0.1.10
natsort                       8.4.0
ndindex                       1.7
nest-asyncio                  1.5.8
networkx                      3.2.1
neurom                        3.2.5
nibabel                       5.2.0
npe2                          0.7.3
nptyping                      2.5.0
numba                         0.58.1
numexpr                       2.8.8
numpy                         1.26.3
numpydoc                      1.5.0
nvidia-ml-py                  12.535.133
oauthlib                      3.2.2
opt-einsum                    3.3.0
packaging                     23.2
pandas                        2.1.4
parso                         0.8.3
partd                         1.4.1
pexpect                       4.9.0
pillow                        10.2.0
Pint                          0.23
pip                           23.3.2
platformdirs                  4.1.0
pooch                         1.8.0
prompt-toolkit                3.0.43
protobuf                      3.19.6
psutil                        5.9.7
psygnal                       0.9.5
ptyprocess                    0.7.0
pure-eval                     0.2.2
py                            1.11.0
py-cpuinfo                    9.0.0
pyasn1                        0.5.1
pyasn1-modules                0.3.0
pyconify                      0.1.6
pydantic                      1.10.13
pydantic-compat               0.1.2
Pygments                      2.17.2
pyinspect                     0.1.0
pynrrd                        1.0.0
PyOpenGL                      3.1.7
pyparsing                     3.1.1
pyproject_hooks               1.0.0
PyQt5                         5.15.10
PyQt5-Qt5                     5.15.2
PyQt5-sip                     12.13.0
python-dateutil               2.8.2
pytz                          2023.3.post1
PyYAML                        6.0.1
pyzmq                         25.1.2
qtconsole                     5.5.1
QtPy                          2.4.1
referencing                   0.32.1
requests                      2.31.0
requests-oauthlib             1.3.1
retry                         0.9.2
rich                          13.7.0
rpds-py                       0.16.2
rsa                           4.9
scikit-image                  0.22.0
scikit-learn                  1.3.2
scipy                         1.11.4
setuptools                    69.0.3
six                           1.16.0
slurmio                       0.1.1
snowballstemmer               2.2.0
soupsieve                     2.5
Sphinx                        4.5.0
sphinxcontrib-applehelp       1.0.4
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.1
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
stack-data                    0.6.3
superqt                       0.6.1
tables                        3.9.2
tensorboard                   2.11.2
tensorboard-data-server       0.6.1
tensorboard-plugin-wit        1.8.1
tensorflow                    2.11.1
tensorflow-estimator          2.11.0
tensorflow-io-gcs-filesystem  0.35.0
termcolor                     2.4.0
threadpoolctl                 3.2.0
tifffile                      2023.12.9
tomli                         2.0.1
tomli_w                       1.0.0
toolz                         0.12.0
tornado                       6.4
tqdm                          4.66.1
traitlets                     5.14.1
traittypes                    0.2.1
treelib                       1.7.0
typer                         0.9.0
typing_extensions             4.9.0
tzdata                        2023.4
urllib3                       2.1.0
vedo                          2023.5.0
vispy                         0.12.2
vtk                           9.3.0
wcwidth                       0.2.13
Werkzeug                      3.0.1
wheel                         0.42.0
widgetsnbextension            4.0.9
wrapt                         1.16.0
zipp                          3.17.0

[adamltyson]

Hi @Eddymorphling,

It looks like something went wrong with our release process. This was fixed in the latest version, but that version wasn't actually released on PyPI! I'm in the process of releasing v1.1.2 and this should be on PyPI in the next hour. Could you test with pip install brainglobe-workflows -U when you get a chance?

[Eddymorphling]

Hi @adamltyson . Thanks for letting me know. I did update brainglobe-workflows and currently sit on v1.1.3. But when I run the same CLI for brainmapper, I do get another import error as below:

ImportError: cannot import name 'check_unique_list' from 'brainglobe_utils.general.list' (/home/conda/envs/brainglobe_v1/lib/python3.10/site-packages/brainglobe_utils/general/list.py)

[adamltyson]

Can you try pip install brainglobe-utils -U?

In general I think (hope) many of these issues can be solved with a fresh installation in a new environment. Please bear with us while we shuffle things around.

[Eddymorphling]

Not a problem at all, I hope my posts are not bothering you too much! Yes, updating brainglobe-utilsworks. I will keep an eye out for PyPI updates moving forwards. Thanks again!

[adamltyson]

Not a problem at all, I hope my posts are not bothering you too much!

Not at all! Please raise any and all issues you find. There's always something that slips through the net.

[Eddymorphling]

@adamltyson Just a naive q about the brainmapperCLI command line. If I am using a self-trained model during cell detection using the brainmapperCLI, how do I tell the job to use "pre-trained weights" during cell detection/classification. Similar to the "use-pretrained-weights" option in the cellfinder-napari interface (screenshot below)

[adamltyson]

If you've trained a model, you just use that. Using pre-trained weights means using the default model the software ships with.

As an aside, if you have new questions (rather than a bug report), could you ask on the image.sc forum, tagging your post with brainglobe? This way others who may have a similar question can more easily find the answer. Thanks!