knift template matching build index file get empty

[Pradyumna-jain]

System information

• OS - WSL-> Ubuntu 18.04.5
• Bazel version: bazel 5.2.0
• OpenCV version : 4

Describe the problem:

Hi when i try to build my template matching index file as below: GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/template_matching/template_matching_tflite --calculator_graph_config_file=mediapipe/graphs/template_matching/index_building.pbtxt --input_side_packets="file_directory=,file_suffix='png',output_index_filename="

i put the template image in the directory (the attach file is the template image) and the output index file is a empty file with 0 byte no error message

Complete Logs:

max_queue_size limits the number of packets enqueued on any input stream

by throttling inputs to the graph. This makes the graph only process one

frame per time.

max_queue_size: 1

Decodes an input video file into images and a video header.

node { calculator: "LocalFilePatternContentsCalculator" input_side_packet: "FILE_DIRECTORY:file_directory" input_side_packet: "FILE_SUFFIX:file_suffix" output_stream: "CONTENTS:encoded_image" }

node { calculator: "OpenCvEncodedImageToImageFrameCalculator" input_stream: "encoded_image" output_stream: "image_frame" }

node: { calculator: "ImageTransformationCalculator" input_stream: "IMAGE:image_frame" output_stream: "IMAGE:scaled_image_frame" node_options: { [type.googleapis.com/mediapipe.ImageTransformationCalculatorOptions] { output_width: 320 output_height: 320 scale_mode: FILL_AND_CROP } } }

node { calculator: "ImagePropertiesCalculator" input_stream: "IMAGE:scaled_image_frame" output_stream: "SIZE:input_video_size" }

node { calculator: "FeatureDetectorCalculator" input_stream: "IMAGE:scaled_image_frame" output_stream: "FEATURES:features" output_stream: "LANDMARKS:landmarks" output_stream: "PATCHES:patches" node_options: { [type.googleapis.com/mediapipe.FeatureDetectorCalculatorOptions] { max_features: 400 } } }

input tensors: 2003232*1 float

output tensors: 20040 float, only first keypoint.size()40 is knift features,

rest is padded by zero.

node { calculator: "TfLiteInferenceCalculator" input_stream: "TENSORS:patches" output_stream: "TENSORS:knift_feature_tensors" input_stream_handler { input_stream_handler: "DefaultInputStreamHandler" } node_options: { [type.googleapis.com/mediapipe.TfLiteInferenceCalculatorOptions] { model_path: "mediapipe/models/knift_float_400.tflite" } } }

node { calculator: "TfLiteTensorsToFloatsCalculator" input_stream: "TENSORS:knift_feature_tensors" output_stream: "FLOATS:knift_feature_floats" }

node { calculator: "BoxDetectorCalculator" input_side_packet: "OUTPUT_INDEX_FILENAME:output_index_filename" input_stream: "FEATURES:features" input_stream: "IMAGE_SIZE:input_video_size" input_stream: "DESCRIPTORS:knift_feature_floats"

node_options: { [type.googleapis.com/mediapipe.BoxDetectorCalculatorOptions] { detector_options { index_type: OPENCV_BF detect_every_n_frame: 1 } } } } I20220907 08:43:50.558111 21422 simple_run_graph_main.cc:122] Initialize the calculator graph. I20220907 08:43:50.571517 21422 simple_run_graph_main.cc:137] Start running the calculator graph. I20220907 08:43:50.603758 21422 simple_run_graph_main.cc:152] Success!

[sureshdagooglecom]

Hi @Pradyumna-jain , Could you answer below questions. Q: can you find any log messages indicating progress throug hgthe set of input images?
Q: are there more than one image in your input set?
If Yes, then let's debug the calculator which write the index.

[Pradyumna-jain]

Hi @sureshdagooglecom . A: No I can't see any log messages. A: I have tried with one image and more than one image but output was same.

[kuaashish]

Hi @Pradyumna-jain, Command line looks suspicious. specifically the "file_directory" assigned empty string "output_index_filename" assigned empty string. let's look at how "output_index_filename" is used in the "template_matching_tflite" binary. Could try out the below steps and build again the solution

Step 1: Put all template images in a single directory.

Step 2: To build the index file for all templates in the directory, run

blaze build -c opt --define DRISHTI_DISABLE_GPU=1 \
 third_party/mediapipe/examples/desktop/template_matching:template_matching_tflite

blaze-bin/third_party/mediapipe/examples/desktop/template_matching/template_matching_tflite \
--calculator_graph_config_file=third_party/mediapipe/graphs/template_matching/index_building.pbtxt \
--input_side_packets="file_directory=<template image directory>,file_suffix=png,output_index_filename=<output index filename>"

The output index file includes the extracted KNIFT features.

Step 3: Replace third_party/mediapipe/models/knift_index.pb with the index file you generated, and update third_party/mediapipe/models/knift_labelmap.txt with your own template names.

Step 4: Build and run the app using the same instructions in Matching US Dollar Bills.

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler[bot]]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No