tensorflow / models

Models and examples built with TensorFlow
Other
76.94k stars 45.8k forks source link

raise TypeError('Expected Operation, Variable, or Tensor, got ' + str(x)) TypeError: Expected Operation, Variable, or Tensor, got level_5 #8886

Open hasansalimkanmaz opened 4 years ago

hasansalimkanmaz commented 4 years ago

Dear,

Recently, i was migrating to tensorflow object detetction API version 2.0 from 1.15.0, Able to train & Evaluate the model. But while creating a frozen model using exporter_main_v2.py getting below error

WARNING:tensorflow:Skipping full serialization of Keras layer <object_detection.meta_architectures.ssd_meta_arch.SSDMetaArch object at 0x7f77e84d3510>, because it is not built. W0716 16:57:39.567866 140159404160768 save_impl.py:76] Skipping full serialization of Keras layer <object_detection.meta_architectures.ssd_meta_arch.SSDMetaArch object at 0x7f77e84d3510>, because it is not built. Traceback (most recent call last): File "models/research/object_detection/exporter_main_v2.py", line 126, in <module> app.run(main) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/absl/app.py", line 299, in run _run_main(main, args) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/absl/app.py", line 250, in _run_main sys.exit(main(argv)) File "models/research/object_detection/exporter_main_v2.py", line 122, in main FLAGS.output_directory) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/object_detection/exporter_lib_v2.py", line 180, in export_inference_graph signatures=concrete_function) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/saved_model/save.py", line 951, in save obj, export_dir, signatures, options, meta_graph_def) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/saved_model/save.py", line 1022, in _build_meta_graph _ = _SaveableView(checkpoint_graph_view) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/saved_model/save.py", line 173, in __init__ self.checkpoint_view.objects_ids_and_slot_variables()) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/training/tracking/graph_view.py", line 415, in objects_ids_and_slot_variables trackable_objects, path_to_root = self._breadth_first_traversal() File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/training/tracking/graph_view.py", line 199, in _breadth_first_traversal for name, dependency in self.list_dependencies(current_trackable): File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/saved_model/save.py", line 108, in list_dependencies extra_dependencies = self.list_extra_dependencies(obj) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/saved_model/save.py", line 137, in list_extra_dependencies self._serialization_cache) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 2746, in _list_extra_dependencies_for_serialization .list_extra_dependencies_for_serialization(serialization_cache)) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/base_serialization.py", line 74, in list_extra_dependencies_for_serialization return self.objects_to_serialize(serialization_cache) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 73, in objects_to_serialize serialization_cache).objects_to_serialize) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 92, in _get_serialized_attributes serialization_cache) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/model_serialization.py", line 53, in _get_serialized_attributes_internal serialization_cache)) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 101, in _get_serialized_attributes_internal functions = save_impl.wrap_layer_functions(self.obj, serialization_cache) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 163, in wrap_layer_functions '{}_layer_call_and_return_conditional_losses'.format(layer.name)) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 503, in add_function self.add_trace(*self._input_signature) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 418, in add_trace trace_with_training(True) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 416, in trace_with_training fn.get_concrete_function(*args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 547, in get_concrete_function return super(LayerCall, self).get_concrete_function(*args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 959, in get_concrete_function concrete = self._get_concrete_function_garbage_collected(*args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 865, in _get_concrete_function_garbage_collected self._initialize(args, kwargs, add_initializers_to=initializers) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 506, in _initialize *args, **kwds)) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2446, in _get_concrete_function_internal_garbage_collected graph_function, _, _ = self._maybe_define_function(args, kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2777, in _maybe_define_function graph_function = self._create_graph_function(args, kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2667, in _create_graph_function capture_by_value=self._capture_by_value), File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 981, in func_graph_from_py_func func_outputs = python_func(*func_args, **func_kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 441, in wrapped_fn return weak_wrapped_fn().__wrapped__(*args, **kwds) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 524, in wrapper ret = method(*args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 170, in wrap_with_training_arg lambda: replace_training_and_call(False)) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 65, in smart_cond pred, true_fn=true_fn, false_fn=false_fn, name=name) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/framework/smart_cond.py", line 54, in smart_cond return true_fn() File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 169, in <lambda> lambda: replace_training_and_call(True), File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 165, in replace_training_and_call return wrapped_call(*args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 566, in call_and_return_conditional_losses return layer_call(inputs, *args, **kwargs), layer.get_losses_for(inputs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/object_detection/meta_architectures/ssd_meta_arch.py", line 251, in call return self._extract_features(inputs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/object_detection/models/ssd_efficientnet_bifpn_feature_extractor.py", line 224, in _extract_features list(zip(self._output_layer_alias, base_feature_maps))) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 968, in __call__ outputs = self.call(cast_inputs, *args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 71, in return_outputs_and_add_losses outputs, losses = fn(inputs, *args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 170, in wrap_with_training_arg lambda: replace_training_and_call(False)) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 65, in smart_cond pred, true_fn=true_fn, false_fn=false_fn, name=name) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/framework/smart_cond.py", line 54, in smart_cond return true_fn() File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 169, in <lambda> lambda: replace_training_and_call(True), File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/utils.py", line 165, in replace_training_and_call return wrapped_call(*args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 541, in __call__ self.call_collection.add_trace(*args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 418, in add_trace trace_with_training(True) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 416, in trace_with_training fn.get_concrete_function(*args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 547, in get_concrete_function return super(LayerCall, self).get_concrete_function(*args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 959, in get_concrete_function concrete = self._get_concrete_function_garbage_collected(*args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 865, in _get_concrete_function_garbage_collected self._initialize(args, kwargs, add_initializers_to=initializers) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 506, in _initialize *args, **kwds)) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2446, in _get_concrete_function_internal_garbage_collected graph_function, _, _ = self._maybe_define_function(args, kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2777, in _maybe_define_function graph_function = self._create_graph_function(args, kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2667, in _create_graph_function capture_by_value=self._capture_by_value), File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 981, in func_graph_from_py_func func_outputs = python_func(*func_args, **func_kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 441, in wrapped_fn return weak_wrapped_fn().__wrapped__(*args, **kwds) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 524, in wrapper ret = method(*args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 484, in wrap_with_training_arg return call_fn(*args, **kwargs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/saving/saved_model/save_impl.py", line 566, in call_and_return_conditional_losses return layer_call(inputs, *args, **kwargs), layer.get_losses_for(inputs) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1627, in get_losses_for reachable = tf_utils.get_reachable_from_inputs(inputs, losses) File "/media/faktion/fastdisk/hasan/tf_2_entity_extraction_model/tf_2_od_api_env/lib/python3.7/site-packages/tensorflow/python/keras/utils/tf_utils.py", line 140, in get_reachable_from_inputs raise TypeError('Expected Operation, Variable, or Tensor, got ' + str(x)) TypeError: Expected Operation, Variable, or Tensor, got level_5

ravikyram commented 4 years ago

@2008hsk

Can you please share colab ink or provide the exact sequence of commands / steps that you executed before running into the problem.It helps us in localizing the issue faster.Thanks!

hasansalimkanmaz commented 4 years ago

I am training the model locally.

after setting up object detection api.

I trained my model with this command. ` python models/research/object_detection/model_main_tf2.py --pipeline_config_path=models/my_model_dir/my_model.config --model_dir=models/my_model_dir/ --alsologtostderr

Then, I have evaluated model with this command, python models/research/object_detection/model_main_tf2.py --pipeline_config_path=models/my_model_dir/my_model.config --model_dir=models/my_model_dir/ --checkpoint_dir=models/my_model_dir/ --alsologtostderr

After evaluation, I run this command to export model python models/research/object_detection/exporter_main_v2.py --input_type=image_tensor --pipeline_config_path=models/my_model_dir/my_model.config --trained_checkpoint_dir=models/my_model_dir/ --output_directory=output_model_21/

then I see the above error message.

hasansalimkanmaz commented 4 years ago

This issue should not be closed. I only mentioned and directed users with the same error to here.

ricardosilva88 commented 4 years ago

Just adding a quick reference to a workaround, in case of coming here in fist place - https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/issues.html

change2014 commented 4 years ago

use exporter_main_v2.py script only, for exporting the models. By default it exports recent checkpoint present in checkpoint_dir. If you want it to export a particular checkpoint then change model_checkpoint_path in checkpoint file.