karllandheer
commented
2 years ago I should mention that the command sm.set_framework('tf.keras') does not fix this error
Open karllandheer opened 2 years ago
karllandheer
commented
2 years ago I should mention that the command sm.set_framework('tf.keras') does not fix this error
ZFTurbo
commented
2 years ago I added requirements.txt. And checked on my PC - working ok.
tensorflow>=2.3.2
keras_applications>=1.0.8
classification_models_3D>=1.0.3
efficientnet_3D>=1.0.2Please install them and check one more time. Also please uninstall keras:
pip uninstall kerasHello, thank you for your response. I have created a fresh environment with tensorflow==2.6.2. I uninstalled keras and have the aforementioned packages with the correct version. I imported tensorflow.keras as keras, and imported segmentation_models_3D, however I now get a different error:
Traceback (most recent call last):
File "
karllandheer
commented
2 years ago I got the same error using tensorflow==1.15.0
ZFTurbo
commented
2 years ago Can you run file tst_keras.py from repository? What will be the result?
karllandheer
commented
2 years ago Hello, that ran successfully. The missing line that's critical for my environment is:
os.environ["KERAS_BACKEND"] = "tensorflow"
Perhaps that should be mentioned somewhere. Thanks very much for your help
ZFTurbo
commented
2 years ago Thanks for info I will add it in README.
nathanielatom
commented
2 years ago I added requirements.txt. And checked on my PC - working ok.
tensorflow>=2.3.2 keras_applications>=1.0.8 classification_models_3D>=1.0.3 efficientnet_3D>=1.0.2Please install them and check one more time. Also please uninstall keras:
pip uninstall keras
Even after this, on python 3.8 I still get:
ValueError: Invalid axis: 4
from running:
import os
os.environ["KERAS_BACKEND"] = "tensorflow"
import segmentation_models_3D as sm
model1 = sm.Unet('resnet34', encoder_weights='imagenet')Can you post the full error trace?
nathanielatom
commented
2 years ago Sure:
Python 3.8.12 | packaged by conda-forge | (default, Jan 30 2022, 23:53:36)
Type 'copyright', 'credits' or 'license' for more information
IPython 8.1.1 -- An enhanced Interactive Python. Type '?' for help.
In [1]: import os
In [2]: os.environ["KERAS_BACKEND"] = "tensorflow"
In [3]: import segmentation_models_3D as sm
2022-03-05 14:08:43.699751: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2022-03-05 14:08:43.699913: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Segmentation Models: using `tf.keras` framework.
In [4]: model1 = sm.Unet('resnet34', encoder_weights='imagenet')
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Input In [4], in <cell line: 1>()
----> 1 model1 = sm.Unet('resnet34', encoder_weights='imagenet')
File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/segmentation_models_3D/__init__.py:34, in inject_global_submodules.<locals>.wrapper(*args, **kwargs)
32 kwargs['models'] = _KERAS_MODELS
33 kwargs['utils'] = _KERAS_UTILS
---> 34 return func(*args, **kwargs)
File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/segmentation_models_3D/models/unet.py:226, in Unet(backbone_name, input_shape, classes, activation, weights, encoder_weights, encoder_freeze, encoder_features, decoder_block_type, decoder_filters, decoder_use_batchnorm, dropout, **kwargs)
222 else:
223 raise ValueError('Decoder block type should be in ("upsampling", "transpose"). '
224 'Got: {}'.format(decoder_block_type))
--> 226 backbone = Backbones.get_backbone(
227 backbone_name,
228 input_shape=input_shape,
229 weights=encoder_weights,
230 include_top=False,
231 **kwargs,
232 )
234 if encoder_features == 'default':
235 encoder_features = Backbones.get_feature_layers(backbone_name, n=4)
File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/segmentation_models_3D/backbones/backbones_factory.py:102, in BackbonesFactory.get_backbone(self, name, *args, **kwargs)
100 def get_backbone(self, name, *args, **kwargs):
101 model_fn, _ = self.get(name)
--> 102 model = model_fn(*args, **kwargs)
103 return model
File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/classification_models_3D/models_factory.py:74, in ModelsFactory.inject_submodules.<locals>.wrapper(*args, **kwargs)
72 modules_kwargs = self.get_kwargs()
73 new_kwargs = dict(list(kwargs.items()) + list(modules_kwargs.items()))
---> 74 return func(*args, **new_kwargs)
File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/classification_models_3D/models/resnet.py:314, in ResNet34(input_shape, input_tensor, weights, classes, include_top, **kwargs)
313 def ResNet34(input_shape=None, input_tensor=None, weights=None, classes=1000, include_top=True, **kwargs):
--> 314 return ResNet(
315 MODELS_PARAMS['resnet34'],
316 input_shape=input_shape,
317 input_tensor=input_tensor,
318 include_top=include_top,
319 classes=classes,
320 weights=weights,
321 **kwargs
322 )
File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/classification_models_3D/models/resnet.py:231, in ResNet(model_params, input_shape, input_tensor, include_top, classes, weights, **kwargs)
228 init_filters = 64
230 # resnet bottom
--> 231 x = layers.BatchNormalization(name='bn_data', **no_scale_bn_params)(img_input)
232 x = layers.ZeroPadding3D(padding=(3, 3, 3))(x)
233 x = layers.Conv3D(init_filters, (7, 7, 7), strides=(2, 2, 2), name='conv0', **conv_params)(x)
File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:925, in Layer.__call__(self, *args, **kwargs)
919 # Functional Model construction mode is invoked when `Layer`s are called on
920 # symbolic `KerasTensor`s, i.e.:
921 # >> inputs = tf.keras.Input(10)
922 # >> outputs = MyLayer()(inputs) # Functional construction mode.
923 # >> model = tf.keras.Model(inputs, outputs)
924 if _in_functional_construction_mode(self, inputs, args, kwargs, input_list):
--> 925 return self._functional_construction_call(inputs, args, kwargs,
926 input_list)
928 # Maintains info about the `Layer.call` stack.
929 call_context = base_layer_utils.call_context()
File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:1098, in Layer._functional_construction_call(self, inputs, args, kwargs, input_list)
1094 # Use `self._name_scope()` to avoid auto-incrementing the name.
1095 with graph.as_default(), backend.name_scope(self._name_scope()):
1096 # Build layer if applicable (if the `build` method has been
1097 # overridden).
-> 1098 self._maybe_build(inputs)
1099 cast_inputs = self._maybe_cast_inputs(inputs, input_list)
1101 if not self.dynamic:
1102 # Wrapping `call` function in autograph to allow for dynamic control
1103 # flow and control dependencies in call. We are limiting this to
(...)
1106 # tf_convert will respect the value of autograph setting in the
1107 # enclosing tf.function, if any.
File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:2643, in Layer._maybe_build(self, inputs)
2638 if not hasattr(self.build, '_is_default'):
2639 # Any setup work performed only once should happen in an `init_scope`
2640 # to avoid creating symbolic Tensors that will later pollute any eager
2641 # operations.
2642 with tf_utils.maybe_init_scope(self):
-> 2643 self.build(input_shapes) # pylint:disable=not-callable
2644 # We must set also ensure that the layer is marked as built, and the build
2645 # shape is stored since user defined build functions may not be calling
2646 # `super.build()`
2647 Layer.build(self, input_shapes)
File ~/.miniconda/envs/eece571f/lib/python3.8/site-packages/tensorflow/python/keras/layers/normalization.py:301, in BatchNormalizationBase.build(self, input_shape)
299 for x in self.axis:
300 if x < 0 or x >= ndims:
--> 301 raise ValueError('Invalid axis: %d' % x)
302 if len(self.axis) != len(set(self.axis)):
303 raise ValueError('Duplicate axis: %s' % self.axis)
ValueError: Invalid axis: 4Looks pretty similar to the original, coming from keras normalization.py.
$ pip show keras
WARNING: Package(s) not found: keras
$ pip show tensorflow
Name: tensorflow
Version: 2.3.2
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: /home/atom/.miniconda/envs/eece571f/lib/python3.8/site-packages
Requires: absl-py, astunparse, gast, google-pasta, grpcio, h5py, keras-preprocessing, numpy, opt-einsum, protobuf, six, tensorboard, tensorflow-estimator, termcolor, wheel, wrapt
Required-by: segmentation-models-3DCan you also print your versions of:
keras_applications>=1.0.8
classification_models_3D>=1.0.3
efficientnet_3D>=1.0.2keras_applications==1.0.8
classification_models_3D==1.0.3
efficientnet_3D==1.0.2Can you try this:
from classification_models_3D.tfkeras import Classifiers
ResNet18, preprocess_input = Classifiers.get('resnet18')
model = ResNet18(input_shape=(128, 128, 128, 3), weights='imagenet')Yep that works!
In [1]: from classification_models_3D.tfkeras import Classifiers
...: ResNet18, preprocess_input = Classifiers.get('resnet18')
...: model = ResNet18(input_shape=(128, 128, 128, 3), weights='imagenet')
2022-03-07 14:58:02.964417: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2022-03-07 14:58:02.964442: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-03-07 14:58:03.719302: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcuda.so.1
2022-03-07 14:58:03.742182: E tensorflow/stream_executor/cuda/cuda_driver.cc:314] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2022-03-07 14:58:03.742302: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: ||||||||||||||
2022-03-07 14:58:03.742374: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: ||||||||||||||
2022-03-07 14:58:03.742487: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:200] libcuda reported version is: 510.47.3
2022-03-07 14:58:03.742571: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:204] kernel reported version is: 510.47.3
2022-03-07 14:58:03.742639: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:310] kernel version seems to match DSO: 510.47.3
2022-03-07 14:58:03.742835: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-03-07 14:58:03.753121: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 1599960000 Hz
2022-03-07 14:58:03.753790: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x561f279e0040 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2022-03-07 14:58:03.753805: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
Downloading data from https://github.com/ZFTurbo/classification_models_3D/releases/download/v1.0/resnet18_inp_channel_3_tch_0_top_True.h5
135127040/135126440 [==============================] - 13s 0us/stepHello, your example on the main page doesn't show you have to put in the input_shape, however when I do not specify this I get an error. If you do indeed need to specify the input_shape please update the example for future users :)
abdulwaheedsoudagar
commented
1 year ago It was resolved by adding
`tf.keras.backend.set_image_data_format('channels_last')`before importing unet. Also, do not import keras, just use tensorflow.
remyamanikantan
commented
1 year ago ModuleNotFoundError: No module named 'keras.engine' when importing segmentation_models_3D
navyanshmahla
commented
1 year ago ModuleNotFoundError: No module named 'keras.engine' when importing segmentation_models_3D
I also had the same error while I was using tensorflow==2.13.0
I rolled back to version 2.12.0 and it's working all fine for me. This is probably because Tensorflow 2.13.0 changed how we import keras modules which are not part of public APIs. In these cases direct imports won't work. keras.engine is not part of the public API.
There can be two solutions to this: Either roll back to a previous version of tensorflow (I rolled back to 2.12.0) or do the following: I'm stating this from the official git repository of tensorflow releases:
If you were using import keras and you used keras functions that were not public APIs, but were accessible in earlier versions with direct imports. In those cases, please use the following guideline: The API may be available in the public Keras API under a different name, so make sure to look for it on keras.io or TensorFlow docs and switch to the public version.
segmentation-models-3D uses classification-models-3D where you can easily see that it uses keras.engine. Sadly, that hasn't still been changed for newer tf versions. So the best way is to roll back to older versions of tensorflow
remyamanikantan
commented
1 year ago Thank you so much for the solution.
On Sun, Sep 24, 2023, 10:05 AM Navyansh Mahla @.***> wrote:
ModuleNotFoundError: No module named 'keras.engine' when importing segmentation_models_3D
I also had the same error while I was using tensorflow==2.13.0 I rolled back to version 2.12.0 and it's working all fine for me. This is probably because Tensorflow 2.13.0 changed how we import keras modules which are not part of public APIs. In these cases direct imports won't work. keras.engine is not part of the public API.
There can be two solutions to this: Either roll back to a previous version of tensorflow (I rolled back to 2.12.0) or do the following: I'm stating this from the official git repository of tensorflow releases https://github.com/tensorflow/tensorflow/releases/tag/v2.13.0:
If you were using import keras and you used keras functions that were not public APIs, but were accessible in earlier versions with direct imports. In those cases, please use the following guideline:
- The API may be available in the public Keras API under a different name, so make sure to look for it on keras.io or TensorFlow docs and switch to the public version.
segmentation-models-3D uses classification-models-3D where you can easily see that it uses keras.engine. Sadly, that hasn't still been changed for newer tf versions. So the best way is to roll back to previous versions.
— Reply to this email directly, view it on GitHub https://github.com/ZFTurbo/segmentation_models_3D/issues/8#issuecomment-1732525654, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ34NESUKIQZRSSJG4A4UK3X37ZUHANCNFSM5MLNEDQA . You are receiving this because you commented.Message ID: @.***>
Hello, I have the following environments:
tensorflow==1.15.0 and keras==2.3.1 and python==3.6
When I try to run the basic line:
model1 = sm.Unet('resnet34', encoder_weights='imagenet')
I get this error:
File "", line 1, in
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/segmentation_models_3D/init.py", line 34, in wrapper
return func(args, kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/segmentation_models_3D/models/unet.py", line 231, in Unet
kwargs,
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/segmentation_models_3D/backbones/backbones_factory.py", line 102, in get_backbone
model = model_fn(args, *kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/classification_models_3D/models_factory.py", line 74, in wrapper
return func(args, new_kwargs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/classification_models_3D/models/resnet.py", line 321, in ResNet34
kwargs
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/classification_models_3D/models/resnet.py", line 231, in ResNet
x = layers.BatchNormalization(name='bn_data', **no_scale_bn_params)(img_input)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 824, in call
self._maybe_build(inputs)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 2146, in _maybe_build
self.build(input_shapes)
File "/Users/karl.landheer/opt/anaconda3/envs/Seg3D/lib/python3.6/site-packages/tensorflow_core/python/keras/layers/normalization.py", line 289, in build
raise ValueError('Invalid axis: %d' % x)
I have tried many permutations of different versions of keras and tensorflow, but to no avail. Any ideas?