Closed RickZLLiao closed 5 years ago
My Config was as follow:
model { ssd { num_classes: 1 box_coder { faster_rcnn_box_coder { y_scale: 10.0 x_scale: 10.0 height_scale: 5.0 width_scale: 5.0 } } matcher { argmax_matcher { matched_threshold: 0.5 unmatched_threshold: 0.5 ignore_thresholds: false negatives_lower_than_unmatched: true force_match_for_each_row: true } } similarity_calculator { iou_similarity { } } anchor_generator { ssd_anchor_generator { num_layers: 6 min_scale: 0.2 max_scale: 0.95 aspect_ratios: 1.0 aspect_ratios: 2.0 aspect_ratios: 0.5 aspect_ratios: 3.0 aspect_ratios: 0.3333 } } image_resizer { fixed_shape_resizer { height: 256 width: 256 } } box_predictor { convolutional_box_predictor { min_depth: 0 max_depth: 0 num_layers_before_predictor: 0 use_dropout: false dropout_keep_probability: 0.8 kernel_size: 1 box_code_size: 4 apply_sigmoid_to_scores: false conv_hyperparams { activation: RELU_6, regularizer { l2_regularizer { weight: 0.00004 } } initializer { truncated_normal_initializer { stddev: 0.03 mean: 0.0 } } batch_norm { train: true, scale: true, center: true, decay: 0.9997, epsilon: 0.001, } } } } feature_extractor { type: 'ssd_mobilenet_v1' min_depth: 16 depth_multiplier: 0.125 conv_hyperparams { activation: RELU_6, regularizer { l2_regularizer { weight: 0.00004 } } initializer { truncated_normal_initializer { stddev: 0.03 mean: 0.0 } } batch_norm { train: true, scale: true, center: true, decay: 0.9997, epsilon: 0.001, } } } loss { classification_loss { weighted_sigmoid { } } localization_loss { weighted_smooth_l1 { } } hard_example_miner { num_hard_examples: 3000 iou_threshold: 0.99 loss_type: CLASSIFICATION max_negatives_per_positive: 3 min_negatives_per_image: 0 } classification_weight: 1.0 localization_weight: 1.0 } normalize_loss_by_num_matches: true post_processing { batch_non_max_suppression { score_threshold: 1e-8 iou_threshold: 0.6 max_detections_per_class: 100 max_total_detections: 100 } score_converter: SIGMOID } } }
train_config: { batch_size: 1 optimizer { rms_prop_optimizer: { learning_rate: { exponential_decay_learning_rate { initial_learning_rate: 0.004 decay_steps: 800720 decay_factor: 0.95 } } momentum_optimizer_value: 0.9 decay: 0.9 epsilon: 1.0 } } fine_tune_checkpoint: "" from_detection_checkpoint: true
num_steps: 20000 data_augmentation_options { random_horizontal_flip { } } data_augmentation_options { ssd_random_crop { } } }
train_input_reader: { tf_record_input_reader { input_path: "I:/work/Other/L3C_Glass_skew/Model_256_256_DM_P125/data/train.record" } label_map_path: "I:/work/Other/L3C_Glass_skew/Label.pbtxt" }
eval_config: { num_examples: 2000
max_evals: 10 }
eval_input_reader: { tf_record_input_reader { input_path: "I:/work/Other/skew/Model_256_256_DM_P125/data/train.record" } label_map_path: "I:/work/Other/skew/Label.pbtxt" shuffle: false num_readers: 1 }
I have had this issue before, something is wrong with your model. The chip is not able to handle. Based on my web searching, nobody knows what is wrong. It simply fails during execution.
Solution - use a model that works to start with and retrain that one.
Could you please provide the check point file & Mobile SSD V1 config file to me, then I could train it myself. thanks.
You will find what you are looking for here: https://github.com/google/aiyprojects-raspbian/issues/402 and https://github.com/google/aiyprojects-raspbian/issues/314
Today i retrain Embedded MobileNet V1 SSD in the tensorflow 1.10,compiled successful. But i got same problem - Model 'object_detection' is not loaded. How to get rid of it ?
my transfer command was as follow : ./bonnet_model_compiler.par --frozen_graph_path=./Glass_skewEmbeded_MobileNet_SSD_Res256_DM_P125/Glassskew_Embeded_Model_256_256_DM_P125_20180914_1857.pb --output_graph_path=./Glass_skewEmbeded_MobileNet_SSD_Res256_DM_P125/Glassskew_Embeded_Model_256_256_DM_P125_20180914_1857.binaryproto --input_tensor_name="Preprocessor/sub" --output_tensor_names="concat,concat_1" --input_tensor_size=256 --debug
@RickZLLiao @kissste
VisionKit has very limited HW resources and it can only support particular architecture with specific configuration.
When you configure the training pipeline, start with embedded_ssd_mobilenet_v1_coco.config, https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/embedded_ssd_mobilenet_v1_coco.config That one is VisionKit friendly.
We provided more details about your use case, search
How to train and deploy a customized object detection model
at
https://aiyprojects.withgoogle.com/vision/#makers-guide
zhoujustin@ provided a pretrained checkpoint in issue 314
Let me know if this solves the issue or not. Thanks!
@sainttelant Thanks for your help! Would I have to reflesh Image first ?
@sainttelant I test 4 case, (1)python3 test_run_model_on_bonnet.py --model_path /opt/aiy/models/mobilenet_ssd_256res_0.125_person_cat_dog.binaryproto --input_height 256 --input_width 256 ==>Default Model,It's ok. (2)python3 test_run_model_on_bonnet.py --model_path /opt/aiy/models/Glassskew_Embeded_Model_256_256_DM_P125_20180914_1857.binaryproto ==> It's failed. (My training model - Embeded Mobile Model,256x256, depth Multiplier: 0.125 ),Error Messafe as follow... (3)python3 test_run_model_on_bonnet.py --model_path /opt/aiy/models/pikachu_detector.binaryproto --input_height 256 --input_width 256 ==> It's failed. (.binaryproto was from https://github.com/giacomobartoli/vision-kit/tree/master/pikachu-detector) (4)python3 test_run_model_on_bonnet.py --model_path /opt/aiy/models/cat_detector_cwh_180602.binaryproto --input_height 256 --input_width 256 ==>It's failed.(.binaryproto was from chadwallacehart's github ,https://github.com/chadwallacehart/aiy_custom_cat_detector)
Case (2)~(4),has same error , as follow Traceback (most recent call last): File "/opt/aiy/projects-python/src/aiy/_drivers/_spicomm.py", line 128, in transact fcntl.ioctl(self._dev, SPICOMM_IOCTL_TRANSACT, self._tbuf) OSError: [Errno 14] Bad address
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "test_run_model_on_bonnet.py", line 66, in main for i, result in enumerate(camera_inference.run()): File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 76, in run yield self._engine.camera_inference() File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 267, in camera_inference return self._communicate(request).inference_result File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 191, in _communicate response.ParseFromString(self._transport.send(request.SerializeToString())) File "/opt/aiy/projects-python/src/aiy/_drivers/_transport.py", line 33, in send return self._spicomm.transact(request) File "/opt/aiy/projects-python/src/aiy/_drivers/_spicomm.py", line 143, in transact raise SpicommInternalError aiy._drivers._spicomm.SpicommInternalError
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "test_run_model_on_bonnet.py", line 77, in main (i, fps, ','.join(output_tensor_str))) File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 87, in exit self.close() File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 79, in close self._engine.stop_camera_inference() File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 273, in stop_camera_inference self._communicate(request) File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 194, in _communicate raise InferenceException(response.status.message) aiy.vision.inference.InferenceException: StopCameraInference: Camera inference is not running.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test_run_model_on_bonnet.py", line 81, in
Sorry sainttelant,I attach wrong person。 I need your help,weiranzhao.
@sainttelant & @weiranzhao,I need your help.
@RickZLLiao Quick thing to check, are you on the latest SD card image?
@weiranzhao,I'm not update to latestest SD card image. I'll try today, and let you know the result, thanks.
@weiranzhao,After i update the latestest SD card image (aiyprojects-2018-08-03.img),i got errors as follow : (No such file or directory: '/dev/vision_spicomm')
pi@raspberrypi:~/AIY-projects-python/src/examples/vision $ python3 any_model_camera.py --model_path /opt/aiy/models/Glassskew_Embeded_Model_256_256_DM_P125_20180914_1857.binaryproto --input_height 256 --input_width 256
Traceback (most recent call last):
File "any_model_camera.py", line 63, in
@weiranzhao,I try commands as follw, but i got same error ==> sudo apt-get update; sudo apt-get upgrade; sudo reboot ==> dpkg -l aiy* from : https://aiyprojects.withgoogle.com/help/#updating-your-sd-card
@RickZLLiao
That error seems like system issue. Could you try to reflash the sd card with latest image?
@dmitriykovalev
Any idea on the error message after reflashing?
@weiranzhao,After i update the latestest SD card image (aiyprojects-2018-08-03.img) & finishing update,I also found message : ' Failed to start load kernel modules ' , when rebooting in the first line.
@RickZLLiao There is some issue going on with apt-get upgrade. Download the sdcard image from : https://github.com/google/aiyprojects-raspbian/releases/tag/v20180803 and you need not to perform any upgrade on it.
@manoj7410,thanks your responding, i'll try it. And let you know the result.
@weiranzhao @manoj7410 @dmitriykovalev After download the sdcard image from: https://github.com/google/aiyprojects-raspbian/releases/tag/v20180803,i could excute my embedded_mobilenet_ssd model successfully。 I excute cat detection and pikachu detection also successfully. Thanks all your help.
@RickZLLiao Can you share the command which you used to retrain the embedded_mobilnet model please. Thanks
Hi
I retrain with MobileNet SSD V1 with 256x256 DM=0.125,but I got an issue as follw Could anybody solve the issue ? pi@raspberrypi:~/AIY-voice-kit-python/src/examples/vision $ ./object_detection.py --input glassskew_Resized.png --output glassskew_detected.png Traceback (most recent call last): File "/opt/aiy/projects-python/src/aiy/_drivers/_spicomm.py", line 128, in transact fcntl.ioctl(self._dev, SPICOMM_IOCTL_TRANSACT, self._tbuf) OSError: [Errno 14] Bad address
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "./object_detection.py", line 39, in main result = inference.run(image_center) File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 100, in run return self._engine.image_inference(self._key, image, params) File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 327, in image_inference return self._communicate(request).inference_result File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 191, in _communicate response.ParseFromString(self._transport.send(request.SerializeToString())) File "/opt/aiy/projects-python/src/aiy/_drivers/_transport.py", line 33, in send return self._spicomm.transact(request) File "/opt/aiy/projects-python/src/aiy/_drivers/_spicomm.py", line 143, in transact raise SpicommInternalError aiy._drivers._spicomm.SpicommInternalError
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "./object_detection.py", line 50, in
main()
File "./object_detection.py", line 46, in main
image.save(args.output)
File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 110, in exit
self.close()
File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 103, in close
self._engine.unload_model(self._key)
File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 251, in unload_model
self._communicate(request)
File "/opt/aiy/projects-python/src/aiy/vision/inference.py", line 194, in _communicate
raise InferenceException(response.status.message)
aiy.vision.inference.InferenceException: UnloadModel: Model 'object_detection' is not loaded.