microsoft / MMdnn

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.
MIT License
5.78k stars 968 forks source link

Caffe to keras model AttributeError: type object 'h5py.h5.H5PYConfig' has no attribute '__reduce_cython__' #912

Open alecda573 opened 3 years ago

alecda573 commented 3 years ago

Platform : Windows 10

Python version: 3.7.9

Source framework with version: Caffe 1.x Destination framework with version: keras (tensorflow 1.15) Pre-trained model path: OpenPose Body 25 https://github.com/CMU-Perceptual-Computing-Lab/openpose

Running scripts:

I have installed this repo through PyPi using pip install mmdnn

I am trying to convert the model using the following command: mmconvert --srcFramework caffe --inputWeight pose_iter_584000.caffemodel --inputNetwork pose_deploy.prototxt --dstFramework keras --outputModel base.h5

However I get the following error:

File "c:\users\alecd\anaconda3\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\users\alecd\anaconda3\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\alecd\anaconda3\Scripts\mmconvert.exe\__main__.py", line 7, in <module>
  File "c:\users\alecd\anaconda3\lib\site-packages\mmdnn\conversion\_script\convert.py", line 108, in _main
    ret = IRToCode._convert(code_args)
  File "c:\users\alecd\anaconda3\lib\site-packages\mmdnn\conversion\_script\IRToCode.py", line 17, in _convert
    from mmdnn.conversion.keras.keras2_emitter import Keras2Emitter
  File "c:\users\alecd\anaconda3\lib\site-packages\mmdnn\conversion\keras\keras2_emitter.py", line 14, in <module>
    from mmdnn.conversion.keras.extra_layers import Scale
  File "c:\users\alecd\anaconda3\lib\site-packages\mmdnn\conversion\keras\extra_layers.py", line 8, in <module>
    from keras.engine import Layer, InputSpec
  File "C:\Users\alecd\AppData\Roaming\Python\Python38\site-packages\keras\__init__.py", line 3, in <module>
    from . import utils
  File "C:\Users\alecd\AppData\Roaming\Python\Python38\site-packages\keras\utils\__init__.py", line 6, in <module>
    from . import conv_utils
  File "C:\Users\alecd\AppData\Roaming\Python\Python38\site-packages\keras\utils\conv_utils.py", line 9, in <module>
    from .. import backend as K
  File "C:\Users\alecd\AppData\Roaming\Python\Python38\site-packages\keras\backend\__init__.py", line 89, in <module>
    from .tensorflow_backend import *
  File "C:\Users\alecd\AppData\Roaming\Python\Python38\site-packages\keras\backend\tensorflow_backend.py", line 5, in <module>
    import tensorflow as tf
  File "c:\users\alecd\anaconda3\lib\site-packages\tensorflow\__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "c:\users\alecd\anaconda3\lib\site-packages\tensorflow\python\__init__.py", line 84, in <module>
    from tensorflow.python import keras
  File "c:\users\alecd\anaconda3\lib\site-packages\tensorflow\python\keras\__init__.py", line 27, in <module>
    from tensorflow.python.keras import models
  File "c:\users\alecd\anaconda3\lib\site-packages\tensorflow\python\keras\models.py", line 24, in <module>
    from tensorflow.python.keras import metrics as metrics_module
  File "c:\users\alecd\anaconda3\lib\site-packages\tensorflow\python\keras\metrics.py", line 37, in <module>
    from tensorflow.python.keras.engine import base_layer
  File "c:\users\alecd\anaconda3\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 59, in <module>
    from tensorflow.python.keras.saving.saved_model import layer_serialization
  File "c:\users\alecd\anaconda3\lib\site-packages\tensorflow\python\keras\saving\saved_model\layer_serialization.py", line 24, in <module>
    from tensorflow.python.keras.saving.saved_model import save_impl
  File "c:\users\alecd\anaconda3\lib\site-packages\tensorflow\python\keras\saving\saved_model\save_impl.py", line 34, in <module>
    from tensorflow.python.keras.saving import saving_utils
  File "c:\users\alecd\anaconda3\lib\site-packages\tensorflow\python\keras\saving\saving_utils.py", line 31, in <module>
    from tensorflow.python.keras.utils.io_utils import ask_to_proceed_with_overwrite
  File "c:\users\alecd\anaconda3\lib\site-packages\tensorflow\python\keras\utils\io_utils.py", line 31, in <module>
    import h5py
  File "c:\users\alecd\anaconda3\lib\site-packages\h5py\__init__.py", line 34, in <module>
    from . import version
  File "c:\users\alecd\anaconda3\lib\site-packages\h5py\version.py", line 17, in <module>
    from . import h5 as _h5
  File "h5py\h5.pyx", line 41, in init h5py.h5
AttributeError: type object 'h5py.h5.H5PYConfig' has no attribute '__reduce_cython__'

I have 2.8 of h5py in my environment

I can remove this error by installing keras 2.3.1 directly in my environment but then it throws the following error:

AttributeError: module 'tensorflow' has no attribute 'placeholder' which is a TF v2.x issue, and can only be bypassed to my knowledge by import tensorflow.compat.v1 and disabling v2 behavior. However, I would rather not have to dig through the source if there is another obvious fix.

I successfully can get the intermediate representation, and I have successfully converted the model to onnx but when I check the ``` onnx model using onnx.checker.check I get the following bad node: ValidationError: Node (prelu4_2) has input size 1 not in range [min=2, max=2].

==> Context: Bad node spec: conv4_2prelu4_2prelu4_2"PRelu* slope

cookieli commented 3 years ago

Thank you for your issures, now mmdnn only support tensorflow1.x . And maybe you have wrong tf version

alecda573 commented 3 years ago

@cookieli my environment had tensorflow1.15 installed and this did not fix the issue....