derekjchow
commented
7 years ago This is working as intended. The exporter script converts all variables into constants when exporting the graph.
Closed protossw512 closed 7 years ago
derekjchow
commented
7 years ago This is working as intended. The exporter script converts all variables into constants when exporting the graph.
enricorotundo
commented
6 years ago 
austinmw
commented
5 years ago @derekjchow so what needs to be changed to use the exported model with TF serving?
wronk
commented
5 years ago I was able to cobble together a hack for the current code version (as of July 2, 2019). Check it out here.
Note that setting --export_as_saved_model True doesn't seem to help. That flag doesn't affect if/how the variables file is exported.
HassanBT
commented
5 years ago Has anyone been able to solve this
System information
What is the top-level directory of the model you are using: ~/workspace/models
Have I written custom code (as opposed to using a stock example script provided in TensorFlow): I am using config files written by myself
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04 And another Linux server
TensorFlow installed from (source or binary): pip install
TensorFlow version (use command below): 1.2.1
Bazel version (if compiling from source):
CUDA/cuDNN version: 8.0/5.1
GPU model and memory: GTX 1060 6GB
Exact command to reproduce: CUDA_VISIBLE_DEVICES=2 python export_inference_graph.py \ --input_type image_tensor \ --pipeline_config_path /scratch2/wangxiny2/workspace/models/object_detection/configs/rfcn_resnet50_car_Jul_20.config \ --checkpoint_path /scratch2/wangxiny2/workspace/models/object_detection/train_car_Jul_20_3/model.ckpt-17586 \ --inference_graph_path serving_model/1 \ --export_as_saved_model True
Describe the problem
I trained some models on my own dataset, both training and evaluation process went well without any problems. I also expoted the model with option "--export_as_saved_model" to be "False" and tried to run in my python script to inference new images, also without any problems. However, I was planing to server the model with tensorflow serving, which means I need to switch "--export_as_saved_model" to be "True" to get model exported as a servable model. It seems the converting process went fine:
And I also got the .pb file with reasonable size. However, the "variables" folder was empty. According to the example provided by tensorflow serving, it should contain files like "variables.data-00000-of-00001" and "variables.index". When I tried to serve the model, tensorflow serving returned the information below:
It seems like the model was not properly served, since "The specified SavedModel has no variables; no checkpoints were restored."
I tried different checkpoint files with different architectures(rfcn-resnet50 and ssd_mobilenet_v1). It seems all have the same problems. I am not sure if I did something wrong or it is a bug.
Source code / logs
One of the config files I am using:
model { faster_rcnn { num_classes: 1 image_resizer { keep_aspect_ratio_resizer { min_dimension: 961 max_dimension: 1199 } } feature_extractor { type: 'faster_rcnn_resnet50' first_stage_features_stride: 8 } first_stage_anchor_generator { grid_anchor_generator { scales: [0.25, 0.5, 1.0, 2.0] aspect_ratios: [0.5, 1.0, 2.0] height_stride: 8 width_stride: 8 } } first_stage_box_predictor_conv_hyperparams { op: CONV regularizer { l2_regularizer { weight: 0.0 } } initializer { truncated_normal_initializer { stddev: 0.01 } } } first_stage_nms_score_threshold: 0.0 first_stage_nms_iou_threshold: 0.8 first_stage_max_proposals: 300 first_stage_localization_loss_weight: 1.0 first_stage_objectness_loss_weight: 1.0 second_stage_box_predictor { rfcn_box_predictor { conv_hyperparams { op: CONV regularizer { l2_regularizer { weight: 0.0 } } initializer { truncated_normal_initializer { stddev: 0.01 } } } crop_height: 10 crop_width: 10 num_spatial_bins_height: 2 num_spatial_bins_width: 2 } } second_stage_post_processing { batch_non_max_suppression { score_threshold: 0.0 iou_threshold: 0.6 max_detections_per_class: 300 max_total_detections: 300 } score_converter: SOFTMAX } second_stage_localization_loss_weight: 1.0 second_stage_classification_loss_weight: 1.0 } }
train_config: { batch_size: 1 num_steps: 45000 keep_checkpoint_every_n_hours: 1 optimizer { momentum_optimizer: { learning_rate: { manual_step_learning_rate { initial_learning_rate: 0.0003 schedule { step: 0 learning_rate: .0003 } schedule { step: 15000 learning_rate: .00003 } schedule { step: 20000 learning_rate: .000003 } } } momentum_optimizer_value: 0.9 } use_moving_average: false } gradient_clipping_by_norm: 10.0 fine_tune_checkpoint: "/scratch2/wangxiny2/workspace/models/object_detection/resnet_v1_50.ckpt" from_detection_checkpoint: false data_augmentation_options { random_horizontal_flip { } } }
train_input_reader: { tf_record_input_reader { input_path: "/scratch2/wangxiny2/workspace/models/object_detection/car_train.record" } label_map_path: "/scratch2/wangxiny2/workspace/models/object_detection/data/car_label_map.pbtxt" }
eval_config: { num_visualizations: 33 num_examples: 33 max_evals: 1 visualization_export_dir: "/scratch2/wangxiny2/workspace/models/object_detection/eval_car_Jul_20_3" }
eval_input_reader: { tf_record_input_reader { input_path: "/scratch2/wangxiny2/workspace/models/object_detection/car_val.record" } label_map_path: "/scratch2/wangxiny2/workspace/models/object_detection/data/car_label_map.pbtxt" shuffle: false num_readers: 1 }