search

tensorflow / models

Models and examples built with TensorFlow
Other
77.05k stars 45.77k forks source link

DeepMAC ModuleNotFoundError: No module named 'tensorflow_text' #9966

Closed anshkumar closed 3 years ago

anshkumar commented 3 years ago

I'm trying to export the trained DeepMAC model for serving. I'm using following command:

export PYTHONPATH=:/home/deploy/models:/home/deploy/models/research:/home/deploy/models/research/slim
export EXPERIMENT_TYPE="deep_mask_head_rcnn_resnetfpn_coco"
export CHECKPOINT_PATH=/home/deploy/ved/pfg/l2/deep_mask/ckpt-70000
export EXPORT_DIR_PATH=/home/deploy/ved/pfg/l2/exported
export CONFIG_FILE_PATH=/home/deploy/ved/pfg/l2/deep_mask_head_rcnn_voc_s49.yaml

python3 ~/models/official/vision/beta/serving/export_saved_model.py --experiment=${EXPERIMENT_TYPE} \
                   --export_dir=${EXPORT_DIR_PATH}/ \
                   --config_file=${CONFIG_FILE_PATH} \
                   --checkpoint_path=${CHECKPOINT_PATH} \
                   --batch_size=4 \
                   --input_image_size=768,768

But I'm getting following error:

Traceback (most recent call last):
  File "/home/deploy/models/official/vision/beta/serving/export_saved_model.py", line 42, in <module>
    from official.common import registry_imports  # pylint: disable=unused-import
  File "/home/deploy/models/official/common/registry_imports.py", line 18, in <module>
    from official.nlp.configs import experiment_configs
  File "/home/deploy/models/official/nlp/configs/experiment_configs.py", line 19, in <module>
    from official.nlp.configs import wmt_transformer_experiments
  File "/home/deploy/models/official/nlp/configs/wmt_transformer_experiments.py", line 22, in <module>
    from official.nlp.data import wmt_dataloader
  File "/home/deploy/models/official/nlp/data/wmt_dataloader.py", line 38, in <module>
    import tensorflow_text as tftxt
ModuleNotFoundError: No module named 'tensorflow_text'

I don't understand why I need to have tensorflow_text for a vision model.

I tried installing tensorflow_text, but then it also installing tensorflow (CPU) with it, but I use GPU version. What to do ?

I'm guessing this export script was not tested with deepmac. So, in the export_saved_model.py, I commented the nlp imports and added following line:

from official.vision.beta.projects.deepmac_maskrcnn.common import registry_imports

I'm trying to export maskrcnn_deepmac with spinenet49 backbone. But I'm getting following error:

Traceback (most recent call last):
  File "/home/deploy/models/official/vision/beta/serving/export_saved_model.py", line 104, in <module>
    app.run(main)
  File "/home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/absl/app.py", line 303, in run
    _run_main(main, args)
  File "/home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/absl/app.py", line 251, in _run_main
    sys.exit(main(argv))
  File "/home/deploy/models/official/vision/beta/serving/export_saved_model.py", line 100, in main
    export_saved_model_subdir='saved_model')
  File "/home/deploy/models/official/vision/beta/serving/export_saved_model_lib.py", line 109, in export_inference_graph
    timestamped=False)
  File "/home/deploy/models/official/core/export_base.py", line 101, in export
    signatures = export_module.get_inference_signatures(function_keys)
  File "/home/deploy/models/official/vision/beta/serving/export_base.py", line 164, in get_inference_signatures
    input_signature)
  File "/home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 1367, in get_concrete_function
    concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)
  File "/home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 1273, in _get_concrete_function_garbage_collected
    self._initialize(args, kwargs, add_initializers_to=initializers)
  File "/home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 764, in _initialize
    *args, **kwds))
  File "/home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 3050, in _get_concrete_function_internal_garbage_collected
    graph_function, _ = self._maybe_define_function(args, kwargs)
  File "/home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 3444, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 3289, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 999, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py", line 672, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/function.py", line 3971, in bound_method_wrapper
    return wrapped_fn(*args, **kwargs)
  File "/home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/func_graph.py", line 986, in wrapper
    raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:

    /home/deploy/models/official/vision/beta/serving/export_base.py:104 inference_from_image_tensors  *
        return self.serve(inputs)
    /home/deploy/models/official/vision/beta/serving/detection.py:123 serve  *
        detections = self.model.call(
    /home/deploy/models/official/vision/beta/modeling/maskrcnn_model.py:182 call  *
        raw_masks = self.mask_head([mask_roi_features, roi_classes])
    /home/deploy/models/official/vision/beta/modeling/heads/instance_heads.py:398 call  *
        x = self._deconv(x)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py:1023 __call__  **
        self._maybe_build(inputs)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py:2625 _maybe_build
        self.build(input_shapes)  # pylint:disable=not-callable
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/keras/layers/convolutional.py:1254 build
        dtype=self.dtype)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py:655 add_weight
        caching_device=caching_device)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py:815 _add_variable_with_custom_getter
        **kwargs_for_getter)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer_utils.py:139 make_variable
        shape=variable_shape if variable_shape else None)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/variables.py:260 __call__
        return cls._variable_v1_call(*args, **kwargs)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/variables.py:221 _variable_v1_call
        shape=shape)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/variables.py:67 getter
        return captured_getter(captured_previous, **kwargs)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py:752 variable_capturing_scope
        lifted_initializer_graph=lifted_initializer_graph, **kwds)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/variables.py:264 __call__
        return super(VariableMetaclass, cls).__call__(*args, **kwargs)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py:293 __init__
        initial_value = initial_value()
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py:87 __call__
        self._checkpoint_position, shape, shard_info=shard_info)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/training/tracking/base.py:122 __init__
        self.wrapped_value.set_shape(shape)
    /home/deploy/miniconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py:1240 set_shape
        (self.shape, shape))

    ValueError: Tensor's shape (2, 2, 64, 128) is not compatible with supplied shape (2, 2, 64, 64)
vighneshbirodkar commented 3 years ago

Tagging @arashwan if he knows what's going on.

arashwan commented 3 years ago

Export module needs to be written for deepmac since the model is built differently than models under main beta folder. We build the maskrcnn model here https://github.com/tensorflow/models/blob/master/official/vision/beta/serving/detection.py#L44 using the main factory function. If you want to export deepmac, you will need to build the model using deepmac model builder instead.

For installing tensorflow_text on gpu, I am tagging @saberkun

saberkun commented 3 years ago

For TF-text, I contact with TF-text team. The tf-text default setup.py should be GPU and to install CPU they can do pip install tensorflow-text[tensorflow_cpu], but if this is an old version of tensorflow text and the default is CPU, they can use pip install tensorflow-text[tensorflow_gpu] if worse comes to worse; just install tensorflow-text, and then install whatever version of tensorflow they want after.

In the current head of the master branch, we added try-except to the tf-text usage, so the error should not happen after that.

anshkumar commented 3 years ago

@arashwan @vighneshbirodkar added the export script in this PR. The majority of the code is same as in https://github.com/tensorflow/models/blob/master/official/vision/beta/serving ,just updated it with deepmac model. This fixes the current export issue.

arashwan commented 3 years ago

We have approved the PR and merged it to master. Thanks for your contribution.

google-ml-butler[bot] commented 3 years ago

Are you satisfied with the resolution of your issue? Yes No