Use Case "Cell Placement Hippocampus" is broken

[alex4200]

The notebook "Cell Placement Hippocampus" regulary fails, will be excluded from automated tests!

Expected behavior

The notebook "Cell Placement Hippocampus" runs without error.

Actual Behavior (please include screenshot if possible)

Depending on the exact version of numpy installed the notebook might fail. For numpy 1.11.2 it runs fine (but shows a DeprecationWarning) while for numpy 1.16.2 the notebook will fail.

DeprecationWarning:

/opt/conda/lib/python3.6/site-packages/brainbuilder/select_region.py:39: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future in_region[:, :, in_region.shape[2] / 2:] = False

Error:

TypeError: slice indices must be integers or None or have an __index__ method

Steps to reproduce the problem

Just run the notebook

How to fix

In the file select_region.py replace the line 39

in_region[:, :, in_region.shape[2] / 2:] = False

with

in_region[:, :, int(in_region.shape[2] / 2):] = False

Optional infrastructural data (user, platform, browser, environment, ...)

Prod version

[jdcourcol]

it is inside BrainBuilder then.

[jdcourcol]

@alex4200 Do you happen to know the version of BrainBuilder ? May be it is an old one that is not really python3 compatible.

[alex4200]

@jdcourcol I think it is '0.2.1dev0'

[jdcourcol]

this is quite old. current version is 0.11.7

[alex4200]

With brainduilder 0.11.7 installed I get an Import Error in the first cell of the notebook:

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-d30b4ac48144> in <module>()
     12 from voxcell import build, math
     13 from voxcellview.widget import VoxcellWidget
---> 14 from brainbuilder.utils import bbp
     15 import numpy as np
     16 from brainbuilder.select_region import select_hemisphere

/opt/conda/lib/python3.6/site-packages/brainbuilder/utils/bbp.py in <module>()
     10 import pandas as pd
     11 
---> 12 from voxcell import CellCollection
     13 
     14 from brainbuilder.exceptions import BrainBuilderError

ImportError: cannot import name 'CellCollection'

[mgeplf]

What version of voxcell is installed?

[alex4200]

@mgeplf 1.5.1.dev0

[mgeplf]

Could you update that to the latest (2.6.2), it's quite a bit out of date.

[alex4200]

@mgeplf With the updated voxcell version I run into other incompatibility issues:

ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-4-7d93643b17c1> in <module>()
     14 from brainbuilder.utils import bbp
     15 import numpy as np
---> 16 from brainbuilder.select_region import select_hemisphere
     17 
     18 cells = CellCollection()

ModuleNotFoundError: No module named 'brainbuilder.select_region'

I think best is when you take a look yourself at the notebook here: https://collab.humanbrainproject.eu/#/collab/77913/nav/529427

[mgeplf]

is select_hemisphere used in the notebook?

[alex4200]

@mgeplf No it is not.

But later in the notebook there is a cell with the following content

region_ids = [id_ for region_name in subregions for id_ in region_map.ids(region_name)]
annotation = VoxelData.load_nrrd(nrrd_file)

creating an error

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-9-783bf6b91044> in <module>()
----> 1 region_ids = [id_ for region_name in subregions for id_ in region_map.ids(region_name)]
      2 
      3 annotation = VoxelData.load_nrrd(nrrd_file)

<ipython-input-9-783bf6b91044> in <listcomp>(.0)
----> 1 region_ids = [id_ for region_name in subregions for id_ in region_map.ids(region_name)]
      2 
      3 annotation = VoxelData.load_nrrd(nrrd_file)

AttributeError: 'RegionMap' object has no attribute 'ids'

Any ideas here? I have no idea what a RegionMap is. Maybe the attributes have changed?

[mgeplf]

Maybe the attributes have changed?

Yup, should work if you replace region_map.ids(region_name) -> region_map.find(region_name, 'name', with_descendants=True) I believe.

[alex4200]

@mgeplf Yes that seems to work.

Next problem:

region_layers_map = {}
for region_name in subregions:
    for rids in region_map.sub(region_id).ids(region_name):
        region_layers_map[rids] = (region_name, )

results in error

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-12-a5dac92c8ef0> in <module>()
      1 region_layers_map = {}
      2 for region_name in subregions:
----> 3     for rids in region_map.sub(region_id).ids(region_name):
      4         region_layers_map[rids] = (region_name, )

AttributeError: 'RegionMap' object has no attribute 'sub'

[mgeplf]

@alex4200 you say:

Depending on the exact version of numpy installed the notebook might fail. For numpy 1.11.2 it runs fine (but shows a DeprecationWarning) while for numpy 1.16.2 the notebook will fail.

Does the rest of the notebook run fine if we pin the numpy version?

[alex4200]

I am in favor of fixing the notebook and not pinning it to some special version. Because then we might end up that each notebook requires a different numpy version, and you always have to install the right version of numpy, and then the rest of the 100's of packages.

And that is exactly the problem: The default version is 1.11.2, but in some other notebook already a newer version is being installed. After that, this notebook fails.

But yes, with 1.11.2 the notebook works fine. But still, I am in favor of fixing this deprecation warning.

[mgeplf]

I am in favor of fixing the notebook and not pinning it to some special version.

I'd like to do that too, but it may require a significant investment of time. I will have a go at hot-fixing the particular issue, and we can see if that work around works. Otherwise, we may have to go the pining route.

[mgeplf]

I think I've fixed everything. @alex4200 can you test it? I would replace the notebooks with the 'production' ones from usecases repo, and clean the virtualenv to make it match whatever a user has by default.

[alex4200]

@mgeplf Where is the notebook I can test? Is it the DEV notebook from the DEV Brain Simulation Platform?

[mgeplf]

The link you shared before: https://collab.humanbrainproject.eu/#/collab/77913/nav/529427 But, like I said, it should be replaced w/ the original.

[alex4200]

@mgeplf The notebook at https://collab.humanbrainproject.eu/#/collab/77913/nav/529427 still fails. Maybe you forgot to save the notebook?

[mgeplf]

Did you replace them with the original?

[alex4200]

Yes, seems to work again. Closing ticket.

[mgeplf]

Thanks for shepherding that though.