search

sign-language-processing / pose-to-video

Render pose sequences as photorealistic videos.
8 stars 6 forks source link

Exception encountered: Could not interpret initializer identifier: {'class_name': 'Orthogonal', 'config': {'gain': 1.0, 'seed': None}} #8

Closed rem0g closed 3 months ago

rem0g commented 3 months ago

Output: 

pose_to_video --type=controlnet --model=sign/sd-controlnet-mediapipe --pose=sign.pose --video=sign_pose.mp4 --processors animatediff simple_upscaler
Loading input pose ...
Generating video ...
pose_to_video.processors.animatediff
pose_to_video.processors.simple_upscaler
2024-05-31 11:08:36.026560: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-05-31 11:08:36.966542: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
0it [00:00, ?it/s]2024-05-31 11:08:38.120834: W tensorflow/core/common_runtime/gpu/gpu_device.cc:2251] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
0it [00:00, ?it/s]
Traceback (most recent call last):
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/ops/operation.py", line 208, in from_config
    return cls(**config)
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/layers/convolutional/conv2d.py", line 107, in __init__
    super().__init__(
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/layers/convolutional/base_conv.py", line 120, in __init__
    self.kernel_initializer = initializers.get(kernel_initializer)
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/initializers/__init__.py", line 118, in get
    raise ValueError(
ValueError: Could not interpret initializer identifier: {'class_name': 'Orthogonal', 'config': {'gain': 1.0, 'seed': None}}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/gomer/.local/bin/pose_to_video", line 8, in <module>
    sys.exit(main())
  File "/home/gomer/.local/lib/python3.10/site-packages/pose_to_video/bin.py", line 62, in main
    for frame in tqdm(frames):
  File "/home/gomer/.local/lib/python3.10/site-packages/tqdm/std.py", line 1182, in __iter__
    for obj in iterable:
  File "/home/gomer/.local/lib/python3.10/site-packages/pose_to_video/processors/simple_upscaler/__init__.py", line 36, in process
    model = tf.keras.models.load_model(model_path)
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/saving/saving_api.py", line 183, in load_model
    return legacy_h5_format.load_model_from_hdf5(
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/legacy/saving/legacy_h5_format.py", line 133, in load_model_from_hdf5
    model = saving_utils.model_from_config(
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/legacy/saving/saving_utils.py", line 85, in model_from_config
    return serialization.deserialize_keras_object(
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/legacy/saving/serialization.py", line 495, in deserialize_keras_object
    deserialized_obj = cls.from_config(
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/models/model.py", line 517, in from_config
    return functional_from_config(
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/models/functional.py", line 517, in functional_from_config
    process_layer(layer_data)
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/models/functional.py", line 497, in process_layer
    layer = saving_utils.model_from_config(
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/legacy/saving/saving_utils.py", line 85, in model_from_config
    return serialization.deserialize_keras_object(
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/legacy/saving/serialization.py", line 504, in deserialize_keras_object
    deserialized_obj = cls.from_config(cls_config)
  File "/home/gomer/.local/lib/python3.10/site-packages/keras/src/ops/operation.py", line 210, in from_config
    raise TypeError(
TypeError: Error when deserializing class 'Conv2D' using config={'name': 'conv2d', 'trainable': True, 'dtype': 'float32', 'filters': 64, 'kernel_size': [5, 5], 'strides': [1, 1], 'padding': 'same', 'data_format': 'channels_last', 'dilation_rate': [1, 1], 'groups': 1, 'activation': 'relu', 'use_bias': True, 'kernel_initializer': {'class_name': 'Orthogonal', 'config': {'gain': 1.0, 'seed': None}}, 'bias_initializer': {'class_name': 'Zeros', 'config': {}}, 'kernel_regularizer': None, 'bias_regularizer': None, 'activity_regularizer': None, 'kernel_constraint': None, 'bias_constraint': None}.

Exception encountered: Could not interpret initializer identifier: {'class_name': 'Orthogonal', 'config': {'gain': 1.0, 'seed': None}}

Versions: 

pip show tensorflow
pip show keras
Name: tensorflow
Version: 2.16.1
Summary: TensorFlow is an open source machine learning framework for everyone.
Home-page: https://www.tensorflow.org/
Author: Google Inc.
Author-email: packages@tensorflow.org
License: Apache 2.0
Location: /home/gomer/.local/lib/python3.10/site-packages
Requires: absl-py, astunparse, flatbuffers, gast, google-pasta, grpcio, h5py, keras, libclang, ml-dtypes, numpy, opt-einsum, packaging, protobuf, requests, setuptools, six, tensorboard, tensorflow-io-gcs-filesystem, termcolor, typing-extensions, wrapt
Required-by: deepface, retina-face, tf_keras
Name: keras
Version: 3.3.3
Summary: Multi-backend Keras.
Home-page: https://github.com/keras-team/keras
Author: Keras team
Author-email: keras-users@googlegroups.com
License: Apache License 2.0
Location: /home/gomer/.local/lib/python3.10/site-packages
Requires: absl-py, h5py, ml-dtypes, namex, numpy, optree, rich
Required-by: deepface, mtcnn, tensorflow

It looks like my current keras doesnt understand the configuration, maybe it's too old of too new?

Python version is 3.10.12

There isnt a version specified in the repo, can you inform me what is the minimum or maximun version required to run the tool?

Thanks!

AmitMY commented 3 months ago

Please see the conclusions from https://github.com/sign-language-processing/pose-to-video/issues/5 If you would like to make a PR to fix this issue for the future, that would be appreciated.