cannot import on a headless system

[thawn]

importing napari_segment_blobs_and_things_with_membranes on a headless system (the TUD cluster) fails with errors about missing GL libraries. I assume this means that this plugin needs a working display?

is there a workaround to make theis plugins work without gl support?

Cheers,

Till

`napari_segment_blobs_and_things_with_membranes` import errors

--------------------------------------------------------------------------- OSError Traceback (most recent call last) Input In [5], in () 4 import matplotlib.pyplot as plt 5 import numpy as np ----> 6 import napari_segment_blobs_and_things_with_membranes as nsbatwm File /app/env/lib/python3.9/site-packages/napari_segment_blobs_and_things_with_membranes/__init__.py:5, in 2 __version__ = "0.3.1" 3 __common_alias__ = "nsbatwm" ----> 5 from napari.types import ImageData, LabelsData 7 from napari_plugin_engine import napari_hook_implementation 9 import numpy as np File /app/env/lib/python3.9/site-packages/napari/types.py:186, in 176 if sys.version_info >= (3, 9): 177 register_type( 178 Future[data_type], # type: ignore 179 choices=_mgui.get_layers_data, (...) 182 ), 183 ) --> 186 _register_types_with_magicgui() File /app/env/lib/python3.9/site-packages/napari/types.py:157, in _register_types_with_magicgui() 153 from concurrent.futures import Future 155 from magicgui import register_type --> 157 from . import layers 158 from .utils import _magicgui as _mgui 160 for _type in (LayerDataTuple, List[LayerDataTuple]): File :1055, in _handle_fromlist(module, fromlist, import_, recursive) File /app/env/lib/python3.9/site-packages/napari/_lazy.py:45, in install_lazy..__getattr__(name) 42 from scipy import stats # noqa: F401 44 if name in submodules: ---> 45 return import_module(f'{module_name}.{name}') 46 elif name in attr_to_modules: 47 submod = import_module(f'{module_name}.{attr_to_modules[name]}') File /app/env/lib/python3.9/importlib/__init__.py:127, in import_module(name, package) 125 break 126 level += 1 --> 127 return _bootstrap._gcd_import(name[level:], package, level) File /app/env/lib/python3.9/site-packages/napari/layers/__init__.py:11, in 8 from importlib import import_module as _imp 10 from ..utils.misc import all_subclasses as _all_subcls ---> 11 from .base import Layer 12 from .image import Image 13 from .labels import Labels File /app/env/lib/python3.9/site-packages/napari/layers/base/__init__.py:1, in ----> 1 from .base import Layer, no_op File /app/env/lib/python3.9/site-packages/napari/layers/base/base.py:15, in 12 import numpy as np 14 from ...utils._dask_utils import configure_dask ---> 15 from ...utils._magicgui import add_layer_to_viewer, get_layers 16 from ...utils.events import EmitterGroup, Event 17 from ...utils.events.event import WarningEmitter File /app/env/lib/python3.9/site-packages/napari/utils/__init__.py:2, in 1 from ._dask_utils import resize_dask_cache ----> 2 from .colormaps import Colormap 3 from .info import citation_text, sys_info 4 from .notebook_display import nbscreenshot File /app/env/lib/python3.9/site-packages/napari/utils/colormaps/__init__.py:2, in 1 from .colorbars import make_colorbar ----> 2 from .colormap import Colormap 3 from .colormap_utils import ( 4 ALL_COLORMAPS, 5 AVAILABLE_COLORMAPS, (...) 16 matplotlib_colormaps, 17 ) File /app/env/lib/python3.9/site-packages/napari/utils/colormaps/colormap.py:11, in 9 from ..translations import trans 10 from .colorbars import make_colorbar ---> 11 from .standardize_color import transform_color 14 class ColormapInterpolationMode(str, Enum): 15 """INTERPOLATION: Interpolation mode for colormaps. 16 17 Selects an interpolation mode for the colormap. (...) 21 bin between by neighboring controls points. 22 """ File /app/env/lib/python3.9/site-packages/napari/utils/colormaps/standardize_color.py:27, in 24 from typing import Any, Callable, Dict, Sequence 26 import numpy as np ---> 27 from vispy.color import ColorArray, get_color_dict, get_color_names 28 from vispy.color.color_array import _string_to_rgb 30 from ..translations import trans File /app/env/lib/python3.9/site-packages/vispy/color/__init__.py:12, in 10 from ._color_dict import get_color_names, get_color_dict # noqa 11 from .color_array import Color, ColorArray ---> 12 from .colormap import (Colormap, BaseColormap, # noqa 13 get_colormap, get_colormaps) # noqa 15 __all__ = ['Color', 'ColorArray', 'Colormap', 'BaseColormap', 16 'get_colormap', 'get_colormaps', 17 'get_color_names', 'get_color_dict'] File /app/env/lib/python3.9/site-packages/vispy/color/colormap.py:14, in 12 from hsluv import hsluv_to_rgb 13 from ..util.check_environment import has_matplotlib ---> 14 import vispy.gloo 16 ############################################################################### 17 # Color maps 18 19 # Length of the texture map used for luminance to RGBA conversion 20 LUT_len = 1024 File /app/env/lib/python3.9/site-packages/vispy/gloo/__init__.py:47, in 5 """ 6 Object oriented interface to OpenGL. 7 (...) 42 43 """ 45 from __future__ import division ---> 47 from . import gl # noqa 48 from .wrappers import * # noqa 49 from .context import (GLContext, get_default_config, # noqa 50 get_current_canvas) # noqa File /app/env/lib/python3.9/site-packages/vispy/gloo/gl/__init__.py:230, in 228 from . import gl2 as default_backend # noqa 229 if default_backend._lib is None: # Probably Android or RPi --> 230 from . import es2 as default_backend # noqa 233 # Call use to start using our default backend 234 use_gl() File /app/env/lib/python3.9/site-packages/vispy/gloo/gl/es2.py:48, in 46 # Else, we failed and exit 47 if es2_file is None: ---> 48 raise OSError('GL ES 2.0 library not found') 49 # Load it 50 _lib = ctypes.CDLL(es2_file) OSError: GL ES 2.0 library not found

[haesleinhuepf]

HI @thawn ,

this bug should be fixed in the recent release of nsbatwm. Can you check? If not, please let me know how I can reproduce the issue.

Thanks! Best, Robert

[haesleinhuepf]

The error appers to still exist. Code:

from skimage.io import imread, imshow, imsave
from napari_segment_blobs_and_things_with_membranes import voronoi_otsu_labeling
import ipywidgets as widgets

import owncloud

Error:

/app/env/lib/python3.9/site-packages/napari_tools_menu/__init__.py:6: UserWarning: importing napari._qt failed
  warnings.warn("importing napari._qt failed")
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
Cell In [3], line 2
      1 from skimage.io import imread, imshow, imsave
----> 2 from napari_segment_blobs_and_things_with_membranes import voronoi_otsu_labeling
      3 import ipywidgets as widgets
      5 import owncloud

File /app/env/lib/python3.9/site-packages/napari_segment_blobs_and_things_with_membranes/__init__.py:15
     13 from skimage.morphology import local_maxima, local_minima
     14 from skimage.restoration import rolling_ball
---> 15 from napari_tools_menu import register_function
     16 from skimage.measure import regionprops
     17 from skimage.segmentation import relabel_sequential

File /app/env/lib/python3.9/site-packages/napari_tools_menu/__init__.py:12
      9 from qtpy.QtWidgets import QMenu
     10 from qtpy.QtCore import QTimer
---> 12 from napari.utils.translations import trans
     13 from toolz import curry
     14 from typing import Callable

File /app/env/lib/python3.9/site-packages/napari/utils/__init__.py:2
      1 from ._dask_utils import resize_dask_cache
----> 2 from .colormaps import Colormap
      3 from .info import citation_text, sys_info
      4 from .notebook_display import nbscreenshot

File /app/env/lib/python3.9/site-packages/napari/utils/colormaps/__init__.py:2
      1 from .colorbars import make_colorbar
----> 2 from .colormap import Colormap
      3 from .colormap_utils import (
      4     ALL_COLORMAPS,
      5     AVAILABLE_COLORMAPS,
   (...)
     17     matplotlib_colormaps,
     18 )

File /app/env/lib/python3.9/site-packages/napari/utils/colormaps/colormap.py:7
      4 import numpy as np
      5 from pydantic import PrivateAttr, validator
----> 7 from napari.utils.color import ColorArray
      9 from ..events import EventedModel
     10 from ..events.custom_types import Array

File /app/env/lib/python3.9/site-packages/napari/utils/color.py:7
      3 from typing import Union
      5 import numpy as np
----> 7 from napari.utils.colormaps.standardize_color import transform_color
     10 class ColorValue(np.ndarray):
     11     """A custom pydantic field type for storing one color value.
     12 
     13     Using this as a field type in a pydantic model means that validation
     14     of that field (e.g. on initialization or setting) will automatically
     15     use the ``validate`` method to coerce a value to a single color.
     16     """

File /app/env/lib/python3.9/site-packages/napari/utils/colormaps/standardize_color.py:27
     24 from typing import Any, Callable, Dict, Sequence
     26 import numpy as np
---> 27 from vispy.color import ColorArray, get_color_dict, get_color_names
     28 from vispy.color.color_array import _string_to_rgb
     30 from ..translations import trans

File /app/env/lib/python3.9/site-packages/vispy/color/__init__.py:12
     10 from ._color_dict import get_color_names, get_color_dict  # noqa
     11 from .color_array import Color, ColorArray
---> 12 from .colormap import (Colormap, BaseColormap,  # noqa
     13                        get_colormap, get_colormaps)  # noqa
     15 __all__ = ['Color', 'ColorArray', 'Colormap', 'BaseColormap',
     16            'get_colormap', 'get_colormaps',
     17            'get_color_names', 'get_color_dict']

File /app/env/lib/python3.9/site-packages/vispy/color/colormap.py:14
     12 from hsluv import hsluv_to_rgb
     13 from ..util.check_environment import has_matplotlib
---> 14 import vispy.gloo
     16 ###############################################################################
     17 # Color maps
     18 
     19 # Length of the texture map used for luminance to RGBA conversion
     20 LUT_len = 1024

File /app/env/lib/python3.9/site-packages/vispy/gloo/__init__.py:47
      5 """
      6 Object oriented interface to OpenGL.
      7 
   (...)
     42 
     43 """
     45 from __future__ import division
---> 47 from . import gl  # noqa
     48 from .wrappers import *  # noqa
     49 from .context import (GLContext, get_default_config,  # noqa
     50                       get_current_canvas)  # noqa

File /app/env/lib/python3.9/site-packages/vispy/gloo/gl/__init__.py:230
    228 from . import gl2 as default_backend  # noqa
    229 if default_backend._lib is None:  # Probably Android or RPi
--> 230     from . import es2 as default_backend  # noqa
    233 # Call use to start using our default backend
    234 use_gl()

File /app/env/lib/python3.9/site-packages/vispy/gloo/gl/es2.py:48
     46 # Else, we failed and exit
     47 if es2_file is None:
---> 48     raise OSError('GL ES 2.0 library not found')
     49 # Load it
     50 _lib = ctypes.CDLL(es2_file)

OSError: GL ES 2.0 library not found

[haesleinhuepf]

napari-tools-menu is version 0.1.17

[thawn]

the error still exists. you can reproduce it in google colab:

https://colab.research.google.com/github/BiAPoL/Bio-image_Analysis_with_Python/blob/main/03_image_processing/simulated_dataset.ipynb

[haesleinhuepf]

Hi @thawn ,

make sure you have the recent version installed. Your Google Colab notebook does not contain any pip install statements (and I failed running it, also because pyclesperanto wasn't installed).

With the current version of nsbatwm it works (in colab).

Try it out like this:

!pip install napari-segment-blobs-and-things-with-membranes==0.3.6

import numpy as np
import napari_segment_blobs_and_things_with_membranes as nsbatwm
binary3 = nsbatwm.split_touching_objects(np.random.random((10,10)).astype(int))

Let me know if the issue persists!

Best, Robert

[thawn]

Hi @haesleinhuepf, I can confirm that the command runs without errors. However, it returns the input data unchanged.

I manually added the pip install command to the notebook I sent you. I will try again with napari-segment-blobs-and-things-with-membranes==0.3.6 and send you a link to a notebook that demonstrates the issue.

[haesleinhuepf]

However, it returns the input data unchanged.

Maybe try

nsbatwm.split_touching_objects(np.random.random((10,10)) > 0.5)

[somas193]

@haesleinhuepf I am trying to run the nsbatwm (0.3.6) based segmentation workflow on TUD's cluster and the package still seems to have an OpenGL dependency which prevents it from running.

[haesleinhuepf]

Hi @somas193 ,

can you please post the error message here and the list of installed packages; in particular those which are mentioned in the error message. Thanks!

[somas193]

@haesleinhuepf here is a screenshot of the error message. The versions of the packages mentioned in the error message are given below: nsbatwm-0.3.6 napari_time_slicer-0.4.9 napari-0.4.18 vispy-0.12.2

[haesleinhuepf]

Just FYI: I'm working on this here: https://github.com/haesleinhuepf/napari-time-slicer/pull/12

But the tests fail, ... it may take some time to fix this...

[haesleinhuepf]

Ok, I managed @somas193 . Would you mind trying to install napari-time-slicer==0.5.0 and executing your code again? If the issue persists, please provide instructions and code so that I can reproduce the issue myself. Thanks!

[somas193]

The same error still pops up @haesleinhuepf! I'll later reproduce the issue in a jupyter notebook which would be a lot easier to share and test. However, I've made it work for now by installing OpenGL in the Singularity container. Our Singularity container uses: https://gitlab.com/nvidia/container-images/cuda/-/blob/master/dist/11.3.1/ubuntu2004/base/Dockerfile as a base and I added the lines apt-get update && apt-get install -y --no-install-recommends \ pkg-config \ libglvnd-dev \ libgl1-mesa-dev \ libegl1-mesa-dev \ libgles2-mesa-dev && \ rm -rf /var/lib/apt/lists/* from the file: https://gitlab.com/nvidia/container-images/opengl/blob/ubuntu20.04/glvnd/devel/Dockerfile to our Singularity config file. The 32-bit OpenGL libraries were excluded due to unavailability. It might happen that some lines from the OpenGL docker file would need to be excluded depending on the structure of the Singularity config file.