search

Calamari-OCR / calamari

Line based ATR Engine based on OCRopy
Apache License 2.0
1.04k stars 209 forks source link

ValueError: A KerasTensor cannot be used as input to a TensorFlow function. #356

Closed MGJamJam closed 3 months ago

MGJamJam commented 3 months ago

Hi!

I am trying to train my own model with calamari. However when I run the training command calamari-train --dataset PAGEXML --files images/*.png

I get the following Error:

ValueError: A KerasTensor cannot be used as input to a TensorFlow function. A KerasTensor is a symbolic placeholder for a shape and dtype, used when constructing Keras Functional models or Keras Functions. You can only use it as input to a Keras layer or a Keras operation (from the namespaces `keras.layers` and `keras.operations`). You are likely doing something like:

x = Input(...)
...
tf_fn(x)  # Invalid.

What you should do instead is wrap `tf_fn` in a layer:

class MyLayer(Layer):
    def call(self, x):
        return tf_fn(x)

x = MyLayer()(x)

I would really appreciate some help here as I have no clue what is causing this error or what I need to do differently. Thanks!

andbue commented 3 months ago

Hi, I've never seen that one before, it might be caused by some version mismatch. Could you please give me some more information (version of tensorflow, calamari, python, maybe the whole stack trace that is printed after the exception is raised)?

MGJamJam commented 3 months ago

Sure, so I installed calamari with Pip as suggested in the documentation. I am using following versions:

Whole Stack trace: 

Resolving input files
Found 6 files in the dataset
Preloading dataset type DataSetMode.TRAIN with size 6
Preloading data: 100%|█████████████████████████████████████████████████████████████| 6/6 [00:00<00:00,  6.01it/s]
Computing codec: 100%|█████████████████████████████████████████████████████████| 6/6 [00:00<00:00, 162360.15it/s]
CODEC: ['', ' ', ',', '-', '.', '1', '7', 'A', 'F', 'G', 'H', 'J', 'K', 'P', 'R', 'S', 'V', 'Z', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'k', 'l', 'm', 'n', 'o', 'p', 'r', 's', 't', 'u', 'v', 'w', 'z', '¬', 'ß', 'ü']
/Users/MgJamJam/calamari_local/.venv/lib/python3.9/site-packages/urllib3/__init__.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020
  warnings.warn(
Traceback (most recent call last):
  File "/Users/MgJamJam/calamari_local/.venv/bin/calamari-train", line 8, in <module>
    sys.exit(main())
  File "/Users/MgJamJam/calamari_local/.venv/lib/python3.9/site-packages/calamari_ocr/scripts/train.py", line 367, in main
    run(args)
  File "/Users/MgJamJam/calamari_local/.venv/lib/python3.9/site-packages/calamari_ocr/scripts/train.py", line 357, in run
    trainer.train(
  File "/Users/MgJamJam/calamari_local/.venv/lib/python3.9/site-packages/calamari_ocr/ocr/trainer.py", line 192, in train
    train_net = backend.create_net(codec, graph_type="train",
  File "/Users/MgJamJam/calamari_local/.venv/lib/python3.9/site-packages/calamari_ocr/ocr/backends/tensorflow_backend/tensorflow_backend.py", line 13, in create_net
    model = TensorflowModel(self.network_proto, graph_type,
  File "/Users/MgJamJam/calamari_local/.venv/lib/python3.9/site-packages/calamari_ocr/ocr/backends/tensorflow_backend/tensorflow_model.py", line 52, in __init__
    self.create_network(self.network_proto.dropout, self.input_data, self.input_length)
  File "/Users/MgJamJam/calamari_local/.venv/lib/python3.9/site-packages/calamari_ocr/ocr/backends/tensorflow_backend/tensorflow_model.py", line 145, in create_network
    lstm_seq_len = K.cast(lstm_seq_len, 'int32')
  File "/Users/MgJamJam/calamari_local/.venv/lib/python3.9/site-packages/keras/src/legacy/backend.py", line 329, in cast
    return tf.cast(x, dtype)
  File "/Users/MgJamJam/calamari_local/.venv/lib/python3.9/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/Users/MgJamJam/calamari_local/.venv/lib/python3.9/site-packages/keras/src/backend/common/keras_tensor.py", line 91, in __tf_tensor__
    raise ValueError(
ValueError: A KerasTensor cannot be used as input to a TensorFlow function. A KerasTensor is a symbolic placeholder for a shape and dtype, used when constructing Keras Functional models or Keras Functions. You can only use it as input to a Keras layer or a Keras operation (from the namespaces `keras.layers` and `keras.operations`). You are likely doing something like:

x = Input(...)
...
tf_fn(x)  # Invalid.

What you should do instead is wrap `tf_fn` in a layer:

class MyLayer(Layer):
    def call(self, x):
        return tf_fn(x)

x = MyLayer()(x)
andbue commented 3 months ago

Thanks! I've not tested it extensively but it seems that in order to work with tensorflow>2.14 we need some changes in our code. Unfortunately, I don't have the time at the moment to fix this. You could either try to install tensorflow 2.14 or use the current calamari 2.2.2 release (which is fixed to tensorflow < 2.7). There might be some problems with these older versions of tensorflow not being available for more recent versions of python. In this case, I would start from a conda environment with python 3.7 or 3.8, install tensorflow < 2.7 and then calamari 2.2.2.

MGJamJam commented 3 months ago

Thank you for testing it and answering so quickly!

Two questions here:

1) I installed the calamari with pip install calamari-ocr without specifying a version, thinking that it would install the latest calamari release. Do you know why calamari v1.0.6 was installed? Is this the expected behaviour?

2) I just uninstalled tensorflow and calamari (pip uninstall ...), and then tried to install calamari again with version 2.2.2 running pip install calamari-ocr==2.2.2 which then resulted in the following error, so it seems that the latest version has also some version conflicts. 

INFO: pip is looking at multiple versions of tfaip to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install calamari-ocr and calamari-ocr==2.2.2 because these package versions have conflicting dependencies.

The conflict is caused by:
    calamari-ocr 2.2.2 depends on tensorflow>=2.4.0
    tfaip 1.2.6 depends on tensorflow<2.7.0 and >=2.4.0

Haven't tried out the conda approach yet, will do so tomorrow.

andbue commented 3 months ago
  1. The problem is that the python version and environment determines which version of tensorflow is available. If there is no tensorflow < 2.7 available, pip checks older versions of calamari and finds that calamari 1.0.6 is the last version that has no version requirement set for tensorflow.
  2. This is not really a version conflict between calamari-ocr and tfaip: ip is simply unable to find a tensorflow<2.7.0 and >=2.4.0. This happens on my system with python 3.12 as well – that's why I use docker images or conda environments with python 3.7 or 3.8 for calamari.
mittagessen commented 3 months ago

On 24/05/15 02:04PM, Andreas Büttner wrote:

  1. The problem is that the python version and environment determines which version of tensorflow is available. If there is no tensorflow < 2.7 available, pip checks older versions of calamari and finds that calamari 1.0.6 is the last version that has no version requirement set for tensorflow.

You can yank those older releases from pypi. They can then still be installed explicitly but they won't be considered by pip per default.

MGJamJam commented 3 months ago

@andbue Could you share a working setup code for a docker image or conda environment with the latest calamari version? I tried your approach with conda, which didn't work out as well. I was using a Mac book with an apple M2 Pro Chip, I know that sometimes this can cause issues, so I tried to install calamari with a notbeook on google colab which also didn't work out. Here the code an the error message (only for calamari-train, calamari-predict works):

## Downgrade python version to 3.8 for calamari to work correctly
!apt-get update -y
!apt-get install python3.8 python3.8-distutils
!update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1
!update-alternatives --config python3
!apt-get install python3-pip
!python3 -m pip install --upgrade pip --user
## install compatible tensorflow version for calamari to work correctly
!pip install tensorflow==2.6
## install latest calamari
!pip install calamari-ocr

Error message when running any calamari-train command: 

Traceback (most recent call last):
  File "/usr/local/bin/calamari-train", line 8, in <module>
    sys.exit(run())
  File "/usr/local/lib/python3.8/dist-packages/calamari_ocr/scripts/train.py", line 17, in run
    main(parse_args())
  File "/usr/local/lib/python3.8/dist-packages/calamari_ocr/scripts/train.py", line 33, in parse_args
    from calamari_ocr.ocr.scenario import CalamariScenario
  File "/usr/local/lib/python3.8/dist-packages/calamari_ocr/ocr/scenario.py", line 4, in <module>
    from tfaip.scenario.scenariobase import (
  File "/usr/local/lib/python3.8/dist-packages/tfaip/scenario/scenariobase.py", line 31, in <module>
    import tensorflow as tf
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/__init__.py", line 41, in <module>
    from tensorflow.python.tools import module_util as _module_util
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/__init__.py", line 40, in <module>
    from tensorflow.python.eager import context
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py", line 32, in <module>
    from tensorflow.core.framework import function_pb2
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/function_pb2.py", line 16, in <module>
    from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/attr_value_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_pb2.py", line 16, in <module>
    from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/resource_handle_pb2.py", line 16, in <module>
    from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/core/framework/tensor_shape_pb2.py", line 36, in <module>
    _descriptor.FieldDescriptor(
  File "/usr/local/lib/python3.8/dist-packages/google/protobuf/descriptor.py", line 621, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
andbue commented 3 months ago

Looks like you're almost there! The error is about the protobuf version not being able to load the TensorBoard files that have been created with an older version of protobuf. I'd try to downgrade as indicated by the error message: pip install protobuf==3.20.3.

If this does not work: Here is a Docker file that did it for me some time ago (installs the current master of this repo):

FROM nvidia/cuda:11.5.1-cudnn8-runtime-ubuntu18.04

ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
    apt-get install -y --no-install-recommends software-properties-common && \
    add-apt-repository -y ppa:ubuntu-toolchain-r/ppa && \
    apt-get purge -y --auto-remove software-properties-common && \
    apt-get install -y --no-install-recommends build-essential && \
    apt-get install -y --no-install-recommends python3.7 python3.7-dev python3-pip python3-setuptools python3-wheel python3-distutils && \
    apt clean && \
    python3.7 -m pip --no-cache-dir install --upgrade pip && \
    python3.7 -m pip --no-cache-dir install https://github.com/Calamari-OCR/calamari/tarball/master && \
    apt-get purge -y --auto-remove build-essential && \
    rm -rf /var/lib/apt/lists/*
MGJamJam commented 3 months ago

It is working now 🥳 🎉 Additionally to the protobuf I also had to reinstall keras with the same version as tensorflow. That's the google colab code that works for me now: 

## Downgrade python version to 3.8 for calamari to work correctly
!apt-get update -y
!apt-get install python3.8 python3.8-distutils
!update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.8 1
!update-alternatives --config python3
!apt-get install python3-pip
!python3 -m pip install --upgrade pip --user
## install compatible tensorflow & keras and protobuf version for calamari to work correctly
!pip install tensorflow==2.6
!pip install keras==2.6
!pip install protobuf==3.20.3
## install latest calamari
!pip install calamari-ocr

Thanks for the help!

andbue commented 3 months ago

Thank you! I copied that in my own notebook https://github.com/andbue/calamari_demo – it seems to work, but GPU support of tensorflow 2.6 depends on CUDA 11 which is again a pain to install on colab. Training from scratch is working with python 3.8, but loading the old models from the calamari_models repo produces the good old "bad marshal data" error. With python 3.7, they are loading just fine. Great to have a working demo for colab again!

MGJamJam commented 3 months ago

Awesome! Thanks for sharing your notebook, will be really helpful for training my model!