google-ai-edge / mediapipe

Cross-platform, customizable ML solutions for live and streaming media.
https://ai.google.dev/edge/mediapipe
Apache License 2.0
26.82k stars 5.09k forks source link

error when runing the YouTube-8M feature extraction graph #289

Closed szyszy315 closed 4 years ago

szyszy315 commented 4 years ago

hello. I'm now trying to extract feature of a video and following https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/youtube8m. I have successfully finished first 4 steps but there is also an error when i run the second script in step 5: GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/extract_yt8m_features \ --calculator_graph_config_file=mediapipe/graphs/youtube8m/feature_extraction.pbtxt \ --input_side_packets=input_sequence_example=/tmp/mediapipe/metadata.pb \ --output_side_packets=output_sequence_example=/tmp/mediapipe/features.pb

it just said there is an error without more details so i don't know what to do next, here is the log:


input_side_packet: "inception3_pca_mean_matrix"
input_side_packet: "inception3_pca_projection_matrix"
input_side_packet: "vggish_pca_mean_matrix"
input_side_packet: "vggish_pca_projection_matrix"
output_side_packet: "sequence_example_to_serialize"

node {
  calculator: "StringToSequenceExampleCalculator"
  input_side_packet: "STRING:input_sequence_example"
  output_side_packet: "SEQUENCE_EXAMPLE:parsed_sequence_example"
}

node {
  calculator: "UnpackMediaSequenceCalculator"
  input_side_packet: "SEQUENCE_EXAMPLE:parsed_sequence_example"
  output_side_packet: "DATA_PATH:input_file"
  output_side_packet: "RESAMPLER_OPTIONS:packet_resampler_options"
  output_side_packet: "AUDIO_DECODER_OPTIONS:audio_decoder_options"
  node_options: {
    [type.googleapis.com/mediapipe.UnpackMediaSequenceCalculatorOptions]: {
      base_packet_resampler_options {
        frame_rate: 1.0
        base_timestamp: 0
      }
      base_audio_decoder_options {
        audio_stream { stream_index: 0 }
      }
    }
  }
}

# Decode the entire video.
node {
  calculator: "OpenCvVideoDecoderCalculator"
  input_side_packet: "INPUT_FILE_PATH:input_file"
  output_stream: "VIDEO:decoded_frames"
}

# Extract the subset of frames we want to keep.
node {
  calculator: "PacketResamplerCalculator"
  input_stream: "decoded_frames"
  output_stream: "sampled_decoded_frames"
  input_side_packet: "OPTIONS:packet_resampler_options"
}

node {
  calculator: "ImageFrameToTensorCalculator"
  input_stream: "sampled_decoded_frames"
  output_stream: "tensor_frame"
}

node {
  calculator: "TensorFlowSessionFromFrozenGraphCalculator"
  output_side_packet: "SESSION:session"
  node_options: {
    [type.googleapis.com/mediapipe.TensorFlowSessionFromFrozenGraphCalculatorOptions]: {
      graph_proto_path: "/tmp/mediapipe/classify_image_graph_def.pb"
      tag_to_tensor_names {
        key: "IMG_UINT8"
        value: "DecodeJpeg:0"
      }
      tag_to_tensor_names {
        key: "INCEPTION_POOL3"
        value: "pool_3/_reshape:0"
      }
    }
  }
}

node {
  calculator: "TensorFlowInferenceCalculator"
  input_side_packet: "SESSION:session"
  input_stream: "IMG_UINT8:tensor_frame"
  output_stream: "INCEPTION_POOL3:inception3_hidden_activation_single_element_batch"
  node_options: {
    [type.googleapis.com/mediapipe.TensorFlowInferenceCalculatorOptions]: {
      signature_name: ""
      batch_size: 1
      add_batch_dim_to_tensors: false
    }
  }
}

# Remove the batch dimension.
node: {
  calculator: "TensorSqueezeDimensionsCalculator"
  input_stream: "inception3_hidden_activation_single_element_batch"
  output_stream: "inception3_hidden_activation"
  node_options: {
    [type.googleapis.com/mediapipe.TensorSqueezeDimensionsCalculatorOptions]: {
      dim: 0
    }
  }
}

node {
  calculator: "TensorToMatrixCalculator"
  input_stream: "TENSOR:inception3_hidden_activation"
  output_stream: "MATRIX:inception3_hidden_activation_matrix"
}

node {
  calculator: "MatrixSubtractCalculator"
  input_stream: "MINUEND:inception3_hidden_activation_matrix"
  input_side_packet: "SUBTRAHEND:inception3_pca_mean_matrix"
  output_stream: "mean_subtracted_inception3_matrix"
}
node {
  calculator: "MatrixMultiplyCalculator"
  input_stream: "mean_subtracted_inception3_matrix"
  input_side_packet: "inception3_pca_projection_matrix"
  output_stream: "pca_inception3_matrix"
}
node {
  calculator: "MatrixToVectorCalculator"
  input_stream: "pca_inception3_matrix"
  output_stream: "pca_inception3_vf"
}

######################## END OF VISUAL ###########################

######################## BEGIN OF AUDIO ##########################
node {
  calculator: "AudioDecoderCalculator"
  input_side_packet: "INPUT_FILE_PATH:input_file"
  input_side_packet: "OPTIONS:audio_decoder_options"
  output_stream: "AUDIO:audio"
  output_stream: "AUDIO_HEADER:audio_header"
}

node {
  calculator: "AddHeaderCalculator"
  input_stream: "DATA:audio"
  input_stream: "HEADER:audio_header"
  output_stream: "media_audio"
}

# Always convert the audio to mono.
node {
  calculator: "AverageTimeSeriesAcrossChannelsCalculator"
  input_stream: "media_audio"
  output_stream: "mono_waveform"
}

node {
  calculator: "RationalFactorResampleCalculator"
  input_stream: "mono_waveform"
  output_stream: "resampled_waveform"
  node_options: {
    [type.googleapis.com/mediapipe.RationalFactorResampleCalculatorOptions] {
      target_sample_rate: 16000.0
    }
  }
}
node {
  calculator: "SpectrogramCalculator"
  input_stream: "resampled_waveform"
  output_stream: "spectrogram_squared_magnitude"
  node_options: {
    [type.googleapis.com/mediapipe.SpectrogramCalculatorOptions] {
      frame_duration_seconds: 0.025
      frame_overlap_seconds: 0.015
      output_type: SQUARED_MAGNITUDE
    }
  }
}
node {
  calculator: "MelSpectrumCalculator"
  # MelSpectrumCalculator expects SQUARED_MAGNITUDE input, but its output is in
  # linear magnitude units.
  input_stream: "spectrogram_squared_magnitude"
  output_stream: "mel_spectrum_magnitude"
  node_options: {
    [type.googleapis.com/mediapipe.MelSpectrumCalculatorOptions] {
      # Follow the 'wideband' or '16kHz' speech convention.
      channel_count: 64
      min_frequency_hertz: 125.0
      max_frequency_hertz: 7500.0
    }
  }
}
node {
  calculator: "StabilizedLogCalculator"
  input_stream: "mel_spectrum_magnitude"
  output_stream: "log_mel_spectrum_magnitude"
  node_options: {
    [type.googleapis.com/mediapipe.StabilizedLogCalculatorOptions] {
      stabilizer: 0.01
    }
  }
}
node {
  calculator: "TimeSeriesFramerCalculator"
  input_stream: "log_mel_spectrum_magnitude"
  output_stream: "log_mel_spectrum_magnitude_with_context"
  node_options: {
    [type.googleapis.com/mediapipe.TimeSeriesFramerCalculatorOptions] {
      frame_duration_seconds: 0.96
      frame_overlap_seconds: -0.04
    }
  }
}
node {
  calculator: "MatrixToTensorCalculator"
  input_stream: "log_mel_spectrum_magnitude_with_context"
  output_stream: "log_mel_spectrum_magnitude_tensor"
  node_options: {
    [type.googleapis.com/mediapipe.MatrixToTensorCalculatorOptions] {
      transpose: true
    }
  }
}

node {
  calculator: "TensorFlowSessionFromFrozenGraphCalculator"
  output_side_packet: "SESSION:vggish_session"
  node_options: {
    [type.googleapis.com/mediapipe.TensorFlowSessionFromFrozenGraphCalculatorOptions]: {
      graph_proto_path: "/tmp/mediapipe/vggish_new.pb"
      tag_to_tensor_names {
        key: "INPUT"
        value: "vggish/input_features:0"
      }
      tag_to_tensor_names {
        key: "VGGISH"
        value: "vggish/fc2/BiasAdd:0"
      }
    }
  }
}

node {
  calculator: "TensorFlowInferenceCalculator"
  input_side_packet: "SESSION:vggish_session"
  input_stream: "INPUT:log_mel_spectrum_magnitude_tensor"
  output_stream: "VGGISH:vggish_tensor"
  node_options: {
    [type.googleapis.com/mediapipe.TensorFlowInferenceCalculatorOptions]: {
      signature_name: ""
      batch_size: 128
    }
  }
}

node {
  calculator: "TensorToMatrixCalculator"
  input_stream: "REFERENCE:log_mel_spectrum_magnitude_with_context"
  input_stream: "TENSOR:vggish_tensor"
  output_stream: "MATRIX:vggish_matrix"
  node_options: {
    [type.googleapis.com/mediapipe.TensorToMatrixCalculatorOptions] {
      time_series_header_overrides {
        num_channels: 128
        num_samples: 1
      }
    }
  }
}

node {
  calculator: "MatrixSubtractCalculator"
  input_stream: "MINUEND:vggish_matrix"
  input_side_packet: "SUBTRAHEND:vggish_pca_mean_matrix"
  output_stream: "mean_subtracted_vggish_matrix"
}
node {
  calculator: "MatrixMultiplyCalculator"
  input_stream: "mean_subtracted_vggish_matrix"
  input_side_packet: "vggish_pca_projection_matrix"
  output_stream: "pca_vggish_matrix"
}
node {
  calculator: "MatrixToVectorCalculator"
  input_stream: "pca_vggish_matrix"
  output_stream: "pca_vggish_vf"
}

# Store the features in the SequenceExample.
node {
  calculator: "PackMediaSequenceCalculator"
  input_side_packet: "SEQUENCE_EXAMPLE:parsed_sequence_example"
  output_side_packet: "SEQUENCE_EXAMPLE:sequence_example_to_serialize"
  input_stream: "FLOAT_FEATURE_RGB:pca_inception3_vf"
  input_stream: "FLOAT_FEATURE_AUDIO:pca_vggish_vf"
}

# Serialize the SequenceExample to a string for storage.
node {
  calculator: "StringToSequenceExampleCalculator"
  input_side_packet: "SEQUENCE_EXAMPLE:sequence_example_to_serialize"
  output_side_packet: "STRING:output_sequence_example"
}

I1203 15:38:51.484107  5960 extract_yt8m_features.cc:103] Initialize the calculator graph.
I1203 15:38:51.487056  5960 extract_yt8m_features.cc:106] Start running the calculator graph.
I1203 15:38:51.487398  5962 unpack_media_sequence_calculator.cc:298] Created AudioDecoderOptions:
audio_stream {
  stream_index: 0
}
start_time: 0
end_time: 7
I1203 15:38:51.487421  5962 unpack_media_sequence_calculator.cc:320] Created PacketResamplerOptions:
[mediapipe.PacketResamplerCalculatorOptions.ext] {
  frame_rate: 1
  base_timestamp: 0
  start_time: 0
  end_time: 7000000
}
2019-12-03 15:38:51.560576: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE3 SSE4.1 SSE4.2 AVX AVX2
I1203 15:38:52.144567  5962 audio_decoder.cc:646] Created audio processor 0x7f65940cab10 for file "/home/zeyu/Desktop/classroom.mp4"
I1203 15:38:52.144847  5962 audio_decoder.cc:374] Opened audio stream (id: 1, channels: 2, sample rate: 44100, time base: 1/44100).
2019-12-03 15:38:53.714342: W external/org_tensorflow/tensorflow/core/framework/op_def_util.cc:371] Op BatchNormWithGlobalNormalization is deprecated. It will cease to work in GraphDef version 9. Use tf.nn.batch_normalization().
2019-12-03 15:38:54.695904: W external/org_tensorflow/tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 201326592 exceeds 10% of system memory.
2019-12-03 15:38:55.475264: W external/org_tensorflow/tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 201326592 exceeds 10% of system memory.
2019-12-03 15:38:55.604924: W external/org_tensorflow/tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 201326592 exceeds 10% of system memory.
2019-12-03 15:38:56.191404: W external/org_tensorflow/tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 201326592 exceeds 10% of system memory.
2019-12-03 15:38:57.156535: W external/org_tensorflow/tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 201326592 exceeds 10% of system memory.
I1203 15:39:21.538525  5960 extract_yt8m_features.cc:108] Gathering output side packets.
E1203 15:39:21.765861  5960 extract_yt8m_features.cc:130] Failed to run the graph: ; Error while writing file: /tmp/mediapipe/features.pb```
jiuqiant commented 4 years ago

The error is from here. Seems like it's a file I/O issue. Not sure if the system is OOM or it doesn't have enough disk space. Can you report more details about your OS? Moreover, can you delete /tmp/mediapipe/features.pb and retry?

szyszy315 commented 4 years ago

@jiuqiant hello, my system is ubuntu 16.04. There is enough disk space and OOM does not lead to this error because i have found the way to get ride of "Allocation of 201326592 exceeds 10% of system memory" but the error is still there .I have retried all other steps for many times and they all works.

szyszy315 commented 4 years ago

@jiuqiant in step 2 mkdir /tmp/mediapipe is aimed to make a directory in the root directory of mediapipe but in my os it creat a folder called mediapipe inside a existing directory called tmp. I think maybe it lead to the error because /tmp/ is a system file and i'm not permitted to write to file inside it.

jiuqiant commented 4 years ago

@jiuqiant in step 2 mkdir /tmp/mediapipe is aimed to make a directory in the root directory of mediapipe but in my os it creat a folder called mediapipe inside a existing directory called tmp. I think maybe it lead to the error because /tmp/ is a system file and i'm not permitted to write to file inside it.

--output_side_packets=output_sequence_example= defines the output file path. You can change the output file path in the command line. Absolute path is more preferred.

szyszy315 commented 4 years ago

@jiuqiant still the same error. it created a file called features.pb in /tmp/mediapipe/ and writed something to it. I think it may only decoded a segment of the video because i tried to run Steps to run the YouTube-8M model inference graph with a local video(https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/youtube8m#steps-to-run-the-youtube-8m-model-inference-graph-with-a-local-video) which need feature.pd and the build succeded but

GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/model_inference \
  --calculator_graph_config_file=mediapipe/graphs/youtube8m/local_video_model_inference.pbtxt \
  --input_side_packets=input_sequence_example_path=/tmp/mediapipe/features.pb,input_video_path=/absolute/path/to/the/local/video/file,output_video_path=/tmp/mediapipe/annotated_video.mp4,segment_size=5,overlap=4

failed said i didn't decode the whole video.

jiuqiant commented 4 years ago

If I understand your comment correctly, /tmp/mediapipe/features.pb is generated? Can you report the file size? Moreover, similiar to https://github.com/google/mediapipe/issues/194#issuecomment-546505284, you can run the following python script to see the content of the features.pb.

import tensorflow as tf

sequence_example = open('/tmp/mediapipe/features.pb', 'rb').read()                           
print(tf.train.SequenceExample.FromString(sequence_example))

For the model inference part, can you show me the error message? The program should just throw a warning if the video is not fully decoded. It should not crash the program. You can find the source code here.

szyszy315 commented 4 years ago

@jiuqiant the size of feature.pb is 32.9 kB (32,882 bytes), and i can successfully run the script here is a part of the content of feature.pb. feature { float_list { value: 2.076115369796753 value: 1.1078059673309326 value: -0.18813468515872955 value: -0.8384913206100464 value: -0.11542361229658127 value: -0.2440829873085022 value: -1.3306479454040527 value: 0.7086437344551086 value: 1.4460822343826294 value: -0.884070098400116 value: 0.4248296618461609 value: -0.13979442417621613 value: 0.9824615120887756 value: 0.5154528617858887 value: -1.5065950155258179 value: -0.73182612657547 value: -0.9751976728439331 value: 1.3784875869750977 value: 0.44272834062576294 value: 0.5880029797554016 value: -1.0954900979995728 value: 0.18842922151088715 value: -0.283181756734848 value: 1.825642704963684 value: 1.159321665763855 value: 0.584626317024231 value: 2.486922025680542 value: 0.1595749855041504 value: -0.004658078774809837 value: 1.1477636098861694 value: 1.0637904405593872 value: 1.108216643333435 value: -1.7783350944519043 value: -0.9082486033439636 value: -0.7650951147079468 value: -1.308105230331421 value: 1.6427212953567505 value: -0.15743005275726318 value: -0.03219599276781082 value: 1.384970784187317 value: -1.3257311582565308 value: 0.030308924615383148 value: -2.8624725341796875 value: 0.17116490006446838 value: -0.22443348169326782 value: -1.5767983198165894 value: -1.429734706878662 value: 1.3550169467926025 value: 0.015885576605796814 value: 0.0877981185913086 value: -0.06826288253068924 value: -0.7793705463409424 value: 2.0656070709228516 value: -0.516409695148468 value: 0.8668118119239807 value: -0.041077546775341034 value: -1.3959709405899048 value: -0.08360891044139862 value: -1.9882359504699707 value: 0.949579119682312 value: -2.0266823768615723 value: -1.0518616437911987 value: 1.216860055923462 value: -1.0414947271347046 value: 2.06475830078125 value: -1.900275468826294 value: -0.2767667770385742 value: -1.614473581314087 value: -0.925095796585083 value: -3.356851577758789 value: 0.8574276566505432 value: 0.9824469089508057 value: 1.5532904863357544 value: -1.303615927696228 value: 1.1339962482452393 value: 1.007266879081726 value: 0.1647041141986847 value: -0.6073731780052185 value: -1.9708610773086548 value: -0.709756076335907 value: -2.327376127243042 value: -0.011305160820484161 value: 0.0936271995306015 value: 0.7555102705955505 value: -0.09514406323432922 value: -1.7542684078216553 value: 2.4782512187957764 value: 1.2873051166534424 value: -0.4111500084400177 value: -1.6801884174346924 value: -0.5894775390625 value: -0.42311108112335205 value: 0.1963639259338379 value: -0.04439619556069374 value: 2.3887453079223633 value: 2.415273904800415 value: -0.7920663952827454 value: -1.2967287302017212 value: -1.0240007638931274 value: 1.57100248336792 value: -1.5513333082199097 value: -3.261763572692871 value: -0.8368674516677856 value: 2.48105788230896 value: -0.8066669702529907 value: -1.6707193851470947 value: -0.11786305904388428 value: 0.5136384963989258 value: 0.926856517791748 value: -0.847081184387207 value: -1.1720112562179565 value: 0.8009216785430908 value: -1.550827980041504 value: -0.9219430088996887 value: -0.7456883192062378 value: 0.8307655453681946 value: 1.8678431510925293 value: 3.582793951034546 value: -0.4868828356266022 value: 2.4303548336029053 value: 0.08972973376512527 value: -1.8924450874328613 value: -1.6072789430618286 value: 0.7497991323471069 value: 0.031924206763505936 value: 0.612067699432373 value: -0.7399752140045166 value: 0.31601041555404663 value: -0.6428033709526062 value: -3.2113521099090576 value: 0.31452426314353943 value: -3.185826063156128 value: -2.122425079345703 value: 1.3172622919082642 value: -0.011581912636756897 value: -0.4665193259716034 value: -0.575229823589325 value: -0.44019705057144165 value: 0.752683162689209 value: 0.09542949497699738 value: -1.8906596899032593 value: -1.0785622596740723 value: 1.2238408327102661 value: -0.4560732841491699 value: 0.943932294845581 value: -0.09849487990140915 value: 1.3175866603851318 value: 1.0762571096420288 value: 0.9753843545913696 value: 2.137068271636963 value: 0.3518795669078827 value: -0.008807189762592316 value: 0.24545150995254517 value: -1.4994237422943115 value: -1.3719347715377808 value: 0.7721972465515137 value: -0.8061539530754089 value: 1.6750097274780273 value: -1.6375633478164673 value: -0.08622197806835175 value: 1.820083737373352 value: 3.42944073677063 value: 1.0887162685394287 value: 1.6489651203155518 value: -0.4037898778915405 value: -0.2327711582183838 value: -1.2390763759613037 value: 0.4883323013782501 value: -0.6950839161872864 value: 1.7811808586120605 value: 1.7405081987380981 value: -0.23283472657203674 value: 0.24460241198539734 value: -1.1376416683197021 value: 0.9089964032173157 value: 3.5345003604888916 value: 1.2678627967834473 value: 1.9292559623718262 value: 0.8725306987762451 value: -1.0869665145874023 value: 0.5660413503646851 value: 0.13393375277519226 value: -0.24879832565784454 value: 0.6325265765190125 value: 0.09257601946592331 value: 1.9454909563064575 value: 0.8988593816757202 value: 0.22737044095993042 value: 0.3642851710319519 value: -0.4734339714050293 value: -2.0749330520629883 value: -0.47319236397743225 value: 0.09216434508562088 value: 1.658652424812317 value: 0.6806251406669617 value: -0.5392337441444397 value: 1.648949384689331 value: 1.4028279781341553 value: -0.19719460606575012 value: 0.14736446738243103 value: 1.493246078491211 value: 1.3111799955368042 value: -0.5843704342842102 value: -0.38041794300079346 value: 2.3881468772888184 value: -0.5753710269927979 value: 0.5786773562431335 value: -1.5756491422653198 value: -1.5164716243743896 value: -0.7557620406150818 value: -0.07720915973186493 value: -0.14232070744037628 value: -0.3186644911766052 value: -0.26369768381118774 value: -1.6858769655227661 value: -0.02637369930744171 value: -0.2754836976528168 value: 0.4108368456363678 value: 0.13638675212860107 value: 1.848016619682312 value: 1.7002733945846558 value: -2.4318244457244873 value: 2.9304964542388916 value: 1.212127685546875 value: 0.3838869631290436 value: -0.8915154933929443 value: -1.9818795919418335 value: 0.8447856903076172 value: -0.4125208854675293 value: -0.8733507394790649 value: -0.986823558807373 value: -0.21005408465862274 value: -0.9300406575202942 value: -1.628851294517517 value: 0.05673721060156822 value: 1.2767287492752075 value: -1.8176496028900146 value: -2.134795904159546 value: -0.5140668749809265 value: 0.27915555238723755 value: 0.26706749200820923 value: 0.9186589121818542 value: 1.3928263187408447 value: -0.3353036642074585 value: 4.319987773895264 value: 1.0011574029922485 value: -0.1646348237991333 value: 1.5651878118515015 value: -0.13341094553470612 value: -0.9453008770942688 value: -1.1091512441635132 value: 1.2046048641204834 value: -1.8501906394958496 value: 1.0726304054260254 value: 1.1842310428619385 value: -0.024764731526374817 value: -0.7953068614006042 value: 0.6875433921813965 value: 0.010663069784641266 value: -0.20165225863456726 value: 0.39678940176963806 value: -0.6475030183792114 value: -1.878673791885376 value: -0.079218290746212 value: -1.8984785079956055 value: -0.36981743574142456 value: -1.1236318349838257 value: 1.4404150247573853 value: -1.3435301780700684 value: 0.33469414710998535 value: 0.08946636319160461 value: 0.8926478624343872 value: -0.8621282577514648 value: -0.5364977121353149 value: -0.5715200304985046 value: -0.0524345338344574 value: 0.5212527513504028 value: -0.036801666021347046 value: 0.7975624203681946 value: 2.055530071258545 value: -0.7584375143051147 value: 0.07001752406358719 value: -3.543323516845703 value: -0.23199038207530975 value: -2.6343026161193848 value: -1.4576836824417114 value: -1.8611493110656738 value: -0.2536201477050781 value: 2.48909592628479 value: -0.04734961315989494 value: 0.5331287384033203 value: -0.671368420124054 value: -0.0707893818616867 value: -2.6089415550231934 value: -0.19841080904006958 value: -1.468470573425293 value: 0.05417454242706299 value: -2.8305346965789795 value: -0.061848171055316925 value: -1.006321907043457 value: 0.25219088792800903 value: 1.4386223554611206 value: 0.579887330532074 value: -0.4711796045303345 value: 0.02917114645242691 value: 1.1335608959197998 value: 1.1308274269104004 value: 2.3994791507720947 value: 2.2572693824768066 value: -1.7117764949798584 value: 1.9838805198669434 value: 2.0533621311187744 value: -0.7716501355171204 value: 0.2622033357620239 value: 0.6035615801811218 value: 2.0593576431274414 value: -0.1605762243270874 value: -1.2871464490890503 value: -2.9039580821990967 value: 2.850698947906494 value: -1.27129328250885 value: -0.42772600054740906 value: -0.8661221265792847 value: -2.9051530361175537 value: 0.8335804343223572 value: 0.9288882613182068 value: -0.6807757019996643 value: 2.3977253437042236 value: -0.5715345144271851 value: -1.8101439476013184 value: -0.629018247127533 value: -0.28434693813323975 value: -1.5168163776397705 value: 0.34159472584724426 value: 0.2774682641029358 value: -0.3267296254634857 value: -0.7374221682548523 value: 1.4254999160766602 value: 0.06263265013694763 value: -2.6425418853759766 value: -1.8985848426818848 value: 1.5762429237365723 value: 0.6307982802391052 value: -1.3759032487869263 value: -0.490688681602478 value: 0.22378122806549072 value: -0.693402111530304 value: -1.695326805114746 value: -2.150895833969116 value: -0.7059016823768616 value: -1.5648071765899658 value: -1.7223933935165405 value: 1.099951982498169 value: 0.7067866921424866 value: -0.5848215222358704 value: -1.5359903573989868 value: 2.0208513736724854 value: 1.3732399940490723 value: 0.9065101146697998 value: -0.3077787160873413 value: -0.4899407923221588 value: 2.199974298477173 value: -0.21433964371681213 value: -1.5456434488296509 value: 1.6424561738967896 value: -1.286812663078308 value: 0.4654316008090973 value: 1.2778654098510742 value: 0.6953151226043701 value: -1.4307490587234497 value: 1.3815690279006958 value: -2.9147303104400635 value: -0.280474454164505 value: 0.00995706021785736 value: -0.44313567876815796 value: 1.01633882522583 value: 0.9478065371513367 value: 0.3741763234138489 value: -0.12752550840377808 value: 1.0716766119003296 value: 0.36232084035873413 value: 0.4079861044883728 value: 0.7223873734474182 value: 0.34057605266571045 value: 0.5880310535430908 value: 0.7112571597099304 value: 0.12893719971179962 value: -2.4549248218536377 value: -0.8952239155769348 value: 1.4323134422302246 value: 1.0494647026062012 value: -1.498865008354187 value: -1.9658281803131104 value: 0.20043067634105682 value: -3.62387752532959 value: -0.8979582786560059 value: -1.5463080406188965 value: -0.8505302667617798 value: -0.5094913244247437 value: 2.2940118312835693 value: -0.9385546445846558 value: 1.328762412071228 value: -0.7265905141830444 value: -0.2145901322364807 value: -0.04741540551185608 value: -0.1488470733165741 value: 1.7708771228790283 value: 1.8799668550491333 value: 1.301935076713562 value: -0.6532919406890869 value: 1.0260339975357056 value: 0.6983137130737305 value: -0.9304280281066895 value: 1.076409101486206 value: -0.9686469435691833 value: -2.054157018661499 value: -0.8036925792694092 value: -1.4463199377059937 value: -0.5630106925964355 value: -0.10041627287864685 value: 0.2565438449382782 value: -2.1981208324432373 value: -1.1467270851135254 value: -0.7692071795463562 value: 1.005334734916687 value: 1.5203325748443604 value: -0.9795114994049072 value: -1.3341177701950073 value: -0.6978042125701904 value: -1.3946025371551514 value: -2.3340437412261963 value: -0.5995053052902222 value: -0.6093590259552002 value: -1.1362119913101196 value: 0.965411901473999 value: -0.5325666666030884 value: 1.5325769186019897 value: -1.2943822145462036 value: -0.10280214250087738 value: 0.851248025894165 value: -1.9097346067428589 value: -0.11335210502147675 value: 1.5964782238006592 value: -0.045951440930366516 value: -0.5869671106338501 value: -2.526352882385254 value: -0.5252227783203125 value: 3.331969738006592 value: -0.328927218914032 value: -0.31322500109672546 value: -0.41437578201293945 value: 1.204060435295105 value: -1.0777133703231812 value: 1.507775902748108 value: -0.3260658085346222 value: 1.928621530532837 value: -1.2279019355773926 value: -0.3879219889640808 value: 0.5551758408546448 value: 1.8628013134002686 value: 2.050586223602295 value: -3.7830770015716553 value: 1.4697057008743286 value: 0.3545951843261719 value: -1.0896382331848145 value: 0.49237409234046936 value: -0.9073858261108398 value: -1.1787737607955933 value: -0.5141935348510742 value: 0.10295286774635315 value: 2.2614545822143555 value: 0.5850622057914734 value: -0.48616135120391846 value: -2.161655902862549 value: -0.45164385437965393 value: 1.1072373390197754 value: -0.86528080701828 value: -2.2031350135803223 value: -0.43436387181282043 value: 0.4707050621509552 value: 1.759995460510254 value: 1.1927800178527832 value: -0.5998307466506958 value: -0.4844651520252228 value: 0.08951327204704285 value: -1.231548547744751 value: 1.0700745582580566 value: -0.7404585480690002 value: -0.5306257009506226 value: -1.053593635559082 value: -1.4943562746047974 value: 0.33376118540763855 value: 0.24344927072525024 value: 0.3685351014137268 value: -0.5092860460281372 value: -0.611930251121521 value: 0.7611799240112305 value: -0.31345120072364807 value: -1.129486322402954 value: -1.9233026504516602 value: 2.781923294067383 value: 0.11787816882133484 value: -0.15942081809043884 value: -1.5138243436813354 value: 0.17782726883888245 value: -1.6063125133514404 value: -0.9038397073745728 value: 0.9823489189147949 value: 2.758408308029175 value: -0.01957005262374878 value: 1.5423431396484375 value: 2.3222010135650635 value: -0.6891860365867615 value: -0.9210190176963806 value: 1.7100516557693481 value: 1.5623365640640259 value: -1.707276701927185 value: 0.6837890148162842 value: -1.0777784585952759 value: -1.6204429864883423 value: -2.059587001800537 value: -0.14859679341316223 value: 0.20182068645954132 value: -3.0875930786132812 value: 1.1198688745498657 value: 1.2911971807479858 value: 0.14077423512935638 value: -1.421191930770874 value: 0.779792845249176 value: 0.859072208404541 value: 3.994802951812744 value: 0.1280965656042099 value: -1.0795221328735352 value: 0.6918175220489502 value: -0.7177660465240479 value: -1.715369701385498 value: -0.23453421890735626 value: 0.36311763525009155 value: 0.5149015188217163 value: 0.8318600654602051 value: 0.22643527388572693 value: 0.13885849714279175 value: -1.276604175567627 value: 2.4378278255462646 value: -0.17550931870937347 value: 0.7171204090118408 value: 0.00713212788105011 value: 0.8381497859954834 value: -0.6088944673538208 value: 0.4728585481643677 value: -0.04492643475532532 value: -0.26995864510536194 value: -1.9193251132965088 value: -3.3169825077056885 value: 0.8015750646591187 value: 0.408490389585495 value: 0.09775852411985397 value: 0.132622629404068 value: -1.9742239713668823 value: -0.33567988872528076 value: 1.2375240325927734 value: -2.184105157852173 value: 0.5942186117172241 value: -0.4967365264892578 value: 0.0779062807559967 value: 1.1705176830291748 value: -0.6791409850120544 value: 1.4983488321304321 value: -0.237945094704628 value: -0.6321359872817993 value: -1.5501048564910889 value: -0.7882075309753418 value: -0.8763176798820496 value: -1.3156052827835083 value: -1.0282502174377441 value: -1.1174026727676392 value: 2.293895959854126 value: -2.11755108833313 value: 0.40874210000038147 value: 2.167431116104126 value: 0.9265503883361816 value: -0.1037089079618454 value: -1.3828001022338867 value: -1.254130244255066 value: -2.329331398010254 value: 1.2428333759307861 value: -0.5488374829292297 value: 0.36707204580307007 value: 1.1477612257003784 value: -0.26736360788345337 value: 0.7133390307426453 value: 1.6296908855438232 value: -1.4705464839935303 value: -0.5046520233154297 value: -0.7684431076049805 value: 0.28728359937667847 value: -1.5011811256408691 value: -0.079474076628685 value: -1.017632246017456 value: 2.0540554523468018 value: -1.6730924844741821 value: 0.060628294944763184 value: -0.061575233936309814 value: -1.0044909715652466 value: 3.257035732269287 value: 1.7926173210144043 value: -2.1903316974639893 value: -0.10165868699550629 value: 0.32420504093170166 value: 0.4139823615550995 value: 0.5033915638923645 value: 1.6441519260406494 value: -0.9672199487686157 value: -2.2941830158233643 value: -1.3591971397399902 value: -0.41222211718559265 value: -1.1566503047943115 value: 1.4126023054122925 value: 0.16003163158893585 value: 1.7526302337646484 value: 0.7163461446762085 value: 0.03353457897901535 value: 0.7607404589653015 value: 1.9751901626586914 value: -1.4662978649139404 value: 0.32588014006614685 value: -2.507495880126953 value: -0.05304610729217529 value: -1.522810935974121 value: 0.43970105051994324 value: 2.196805477142334 value: -1.9686163663864136 value: -0.17659585177898407 value: 1.446294903755188 value: -1.0893762111663818 value: 0.582170844078064 value: -1.2488877773284912 value: -0.9753806591033936 value: 1.6396675109863281 value: -1.1663167476654053 value: 0.5754790306091309 value: 0.5257911682128906 value: 0.1384410858154297 value: -2.1029715538024902 value: -0.07679685950279236 value: -0.07024064660072327 value: 4.212103366851807 value: -0.6996765732765198 value: 0.49270692467689514 value: -0.8458765149116516 value: 1.3353288173675537 value: 0.5400585532188416 value: 0.3836984634399414 value: 1.8173166513442993 value: -1.663107991218567 value: -1.7252086400985718 value: -1.0992074012756348 value: 1.3622026443481445 value: 0.2867245078086853 value: -0.04090416431427002 value: 0.4595750868320465 value: 0.7557359933853149 value: 0.6710014939308167 value: -2.4956390857696533 value: 0.6815018653869629 value: 1.5328012704849243 value: 1.0107685327529907 value: -1.0203531980514526 value: 0.6467156410217285 value: -0.4034999907016754 value: 2.6777799129486084 value: 0.4509180188179016 value: 1.3818395137786865 value: 1.9401068687438965 value: 0.8671821355819702 value: 0.09059834480285645 value: 2.3978803157806396 value: 0.5636197328567505 value: -1.1829209327697754 value: 0.9495004415512085 value: 0.1274394989013672 value: -1.750793218612671 value: 0.6026861667633057 value: -0.1557961106300354 value: 1.6593365669250488 value: -0.7877292633056641 value: 0.9259800910949707 value: 0.22194582223892212 value: 0.33618858456611633 value: 0.7520437240600586 value: -0.9863332509994507 value: 0.43588465452194214 value: -0.12301796674728394 value: 0.39134934544563293 value: -0.6367123126983643 value: 0.1919347643852234 value: -1.0463886260986328 value: -1.2894504070281982 value: -1.5288201570510864 value: 1.658541202545166 value: 0.5257874727249146 value: 0.5543509721755981 value: 0.8607103228569031 value: 0.951469361782074 value: 1.5920846462249756 value: -1.4318946599960327 value: 0.47628602385520935 value: -0.6824831962585449 value: 1.4573286771774292 value: 0.3038623332977295 value: -0.06113210320472717 value: 1.058286428451538 value: -0.08354541659355164 value: -0.5825676321983337 value: -0.08145475387573242 value: 0.2608497142791748 value: 1.242293119430542 value: -0.32986271381378174 value: -0.41835957765579224 value: 1.2640823125839233 value: -0.08283620327711105 value: -0.16441693902015686 value: 1.156849980354309 value: -1.387986660003662 value: -1.4179691076278687 value: 0.8726122379302979 value: -2.169668436050415 value: -0.7791728377342224 value: -1.4190475940704346 value: 0.188334658741951 value: -0.31121960282325745 value: 0.8686013221740723 value: 0.12810254096984863 value: 3.1133041381835938 value: -0.6583575010299683 value: -1.891268014907837 value: -1.71553635597229 value: -1.2640318870544434 value: -2.125417470932007 value: -2.2403831481933594 value: -1.3303781747817993 value: 0.10707536339759827 value: -0.10819298028945923 value: 0.7587528228759766 value: -3.6602096557617188 value: -0.10786467790603638 value: -0.8920912146568298 value: 1.5344736576080322 value: 0.4542205333709717 value: 0.2542000412940979 value: -0.9001966714859009 value: -0.850999653339386 value: 0.2876558005809784 value: 0.5929479598999023 value: 0.2646709084510803 value: 0.8582943677902222 value: 1.227980136871338 value: 0.1025797426700592 value: 0.25967758893966675 value: -0.6584393978118896 value: 1.8413779735565186 value: 0.018914014101028442 value: -0.8523246049880981 value: 1.05805242061615 value: 0.08580684661865234 value: -1.0977952480316162 value: 0.41021043062210083 value: -0.2414032220840454 value: -0.8463214039802551 value: -2.2171831130981445 value: -2.2572672367095947 value: -1.536203384399414 value: 0.662703275680542 value: 2.250798225402832 value: -1.529068112373352 value: 1.1428463459014893 value: -2.21043062210083 value: -0.17448699474334717 value: -0.621275007724762 value: -1.462862253189087 value: 0.35131120681762695 value: 0.17737379670143127 value: -0.4714112877845764 value: -1.0097854137420654 value: 0.08917000889778137 value: 0.6742238402366638 value: 0.9743010997772217 value: -1.0466209650039673 value: -1.05338454246521 value: -1.4871054887771606 value: -0.6949794292449951 value: -0.9471079707145691 value: -1.2720668315887451 value: 0.23235979676246643 value: -0.08154436945915222 value: 1.1909196376800537 value: -0.3748297095298767 value: -0.8733246922492981 value: 1.2948898077011108 value: -0.24685853719711304 value: -0.3131599724292755 value: 0.4706265330314636 value: 1.5272151231765747 value: 1.5599596500396729 value: -1.4874463081359863 value: -1.3331787586212158 value: -1.7727274894714355 value: -0.08877497911453247 value: -1.6576133966445923 value: 0.11239799857139587 value: -0.13919253647327423 value: 0.6881515383720398 value: -0.9482897520065308 value: -0.3039206266403198 value: -0.20627111196517944 value: -1.5109062194824219 value: 0.24225211143493652 value: -0.009042799472808838 value: -2.4743900299072266 value: -1.6658531427383423 value: -0.32486337423324585 value: -0.8449218273162842 value: 1.2172571420669556 value: 1.905389666557312 value: -1.2685644626617432 value: -2.5383994579315186 value: 0.9311712384223938 value: 0.5448826551437378 value: 0.15401095151901245 value: 1.1081100702285767 value: -0.5396357178688049 value: 1.155993938446045 value: 1.2488160133361816 value: 0.8588473200798035 value: 0.44656795263290405 value: 1.171327829360962 value: -0.2690891623497009 value: 1.3756144046783447 value: 0.7742607593536377 value: 2.5948703289031982 value: -0.8630634546279907 value: 0.2800670266151428 value: 1.4807395935058594 value: -1.5012378692626953 value: 0.3438887596130371 value: 0.7298605442047119 value: 0.6276078224182129 value: -1.3623075485229492 value: -1.0935521125793457 value: -1.0911192893981934 value: 0.4304130971431732 value: 0.09266266226768494 value: -1.357710838317871 value: -0.5553179979324341 value: -0.28151288628578186 value: -1.9078547954559326 value: -0.41531217098236084 value: -1.1181950569152832 value: 0.8765799403190613 value: -1.0592482089996338 value: -1.9457340240478516 value: 0.8603631258010864 value: 1.6130988597869873 value: 0.31300288438796997 value: 1.5846459865570068 value: 0.10774415731430054 value: -0.05864184349775314 value: 1.5854884386062622 value: -0.7500016093254089 value: -0.7805182933807373 value: -1.533337116241455 value: -0.19407367706298828 value: -0.5015048980712891 value: -1.0564229488372803 value: 0.7987643480300903 value: 0.6627199053764343 value: -0.03455421328544617 value: -2.043048143386841 value: 0.33498305082321167 value: -0.12174426019191742 value: 2.8576321601867676 value: 0.6044117212295532 value: -2.24442458152771 value: -1.1007792949676514 value: -1.6784660816192627 value: 1.3082897663116455 value: 0.36610108613967896 value: -0.7117820382118225 value: 1.445816993713379 value: 1.212733268737793 value: 0.20808029174804688 value: 0.805008590221405 value: -0.4894181489944458 value: -0.50527024269104 value: -1.087148666381836 value: 1.7483556270599365 value: -1.3845714330673218 value: -0.027503877878189087 value: 0.656522274017334 value: 0.07468867301940918 value: 0.8858073949813843 value: -0.31627973914146423 value: 1.1880061626434326 value: -1.6633497476577759 value: -2.947570323944092 value: -2.1041316986083984 value: 0.065956711769104 value: 1.238690972328186 value: -0.670972466468811 value: 0.4631921052932739 value: -0.22584708034992218 value: -0.014074534177780151 value: 1.2271747589111328 value: -0.1408875286579132 value: -1.0871182680130005 value: -0.37171247601509094 value: 0.3868141174316406 value: -1.3457255363464355 value: -0.141131192445755 value: 0.4967617690563202 value: 0.1685529351234436 value: -1.6076552867889404 value: 1.547536849975586 value: -1.2592315673828125 value: 2.341219425201416 value: -0.19430238008499146 value: 0.6378706097602844 value: -0.5116959810256958 value: -2.6581196784973145 value: -0.3645663261413574 value: -0.749751091003418 value: 0.6051032543182373 value: 1.4998607635498047 value: -1.2030404806137085 value: -0.24497655034065247 value: 0.6152099370956421 value: 0.7808939814567566 value: -1.1130836009979248 value: 0.7788217067718506 value: 0.013284295797348022 value: 0.6142373085021973 value: 1.536675214767456 value: 0.6126294732093811 value: 1.4795689582824707 value: -2.3028745651245117 value: -1.0344455242156982 value: 0.6834166049957275 value: -0.2190404236316681 value: 0.30404895544052124 value: -0.5494284629821777 value: -1.1515576839447021 value: 0.8520990014076233 value: -1.2436439990997314 value: -0.2021297961473465 value: 2.032911539077759 value: -0.20717772841453552 value: 2.3595993518829346 value: 0.042632028460502625 value: 1.0470619201660156 value: 1.260422945022583 value: -0.7738087773323059 value: 2.130340337753296 value: 1.127643346786499 value: -0.22513914108276367 value: 0.545540452003479 value: -0.4573346972465515 value: -0.2560608983039856 value: -3.6818153858184814 value: -0.22341525554656982 value: -0.26421135663986206 value: -3.1989917755126953 value: -0.12157242000102997 value: -1.0877983570098877 value: 2.9311070442199707 value: 1.130705714225769 value: -0.7277622222900391 value: -1.2124369144439697 value: -1.4359900951385498 value: 1.0710351467132568 value: -0.4723031520843506 value: -0.022355198860168457 value: -0.655505359172821 value: 1.9591941833496094 value: 2.8833367824554443 value: 0.8208673596382141 value: -0.6619185209274292 value: 0.368146151304245 value: -0.19281776249408722 value: 0.21288150548934937 value: 3.2281436920166016 value: -2.129258632659912 value: 2.809626817703247 value: 0.8000625371932983 value: -0.1811503767967224 value: -0.4792667031288147 value: -1.347848892211914 value: 0.28895899653434753 value: 1.5758254528045654 value: 0.09923964738845825 value: -0.5243102312088013 value: -0.5460996627807617 value: 0.6530378460884094 value: -0.13622522354125977 value: -0.17016145586967468 value: 0.5283380746841431 value: -1.8515403270721436 value: -1.0958117246627808 value: 0.05082771182060242 value: 2.11309552192688 value: 2.02075457572937 value: -0.5967410802841187 value: -0.017846886068582535 value: -1.2482661008834839 value: -0.8981242179870605 value: -1.0454866886138916 value: 1.0482861995697021 value: -0.5114814043045044 value: -2.2299327850341797 value: 0.6608255505561829 value: -0.4123225510120392 value: 2.909165382385254 value: -0.553383469581604 value: -0.3688572645187378 value: 0.984059751033783 } } } } feature_list { key: "RGB/feature/timestamp" value { feature { int64_list { value: 0 } } feature { int64_list { value: 1000000 } } feature { int64_list { value: 2000000 } } feature { int64_list { value: 3000000 } } feature { int64_list { value: 4000000 } } feature { int64_list { value: 5000000 } } feature { int64_list { value: 6000000 } } } } }

szyszy315 commented 4 years ago

@jiuqiant And when i run this script, the output is here: GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/model_inference \ --calculator_graph_config_file=mediapipe/graphs/youtube8m/local_video_model_inference.pbtxt \ --input_side_packets=input_sequence_example_path=/tmp/mediapipe/features.pb,input_video_path=/home/zeyu/Desktop/classroom.mp4,output_video_path=/tmp/mediapipe/annotated_video.mp4,segment_size=5,overlap=4

`I1204 00:53:43.324546 14636 simple_run_graph_main.cc:105] Get calculator graph config contents: input_side_packet: "input_sequence_example_path" input_side_packet: "input_video_path" input_side_packet: "output_video_path" input_side_packet: "segment_size" input_side_packet: "overlap"

node { calculator: "LocalFileContentsCalculator" input_side_packet: "FILE_PATH:input_sequence_example_path" output_side_packet: "CONTENTS:input_sequence_example" }

node { calculator: "StringToSequenceExampleCalculator" input_side_packet: "STRING:input_sequence_example" output_side_packet: "SEQUENCE_EXAMPLE:parsed_sequence_example" }

node { calculator: "UnpackMediaSequenceCalculator" input_side_packet: "SEQUENCE_EXAMPLE:parsed_sequence_example" output_stream: "FLOAT_FEATURE_RGB:rgb_feature_vector" output_stream: "FLOAT_FEATURE_AUDIO:audio_feature_vector" }

node { calculator: "ConcatenateFloatVectorCalculator" input_stream: "rgb_feature_vector" input_stream: "audio_feature_vector" output_stream: "feature_vector" }

node { calculator: "VectorFloatToTensorCalculator" input_stream: "feature_vector" output_stream: "feature_tensor" }

node { calculator: "StringToInt32Calculator" input_side_packet: "segment_size" output_side_packet: "segment_size_int" }

node { calculator: "StringToInt32Calculator" input_side_packet: "overlap" output_side_packet: "overlap_int" }

node { calculator: "LappedTensorBufferCalculator" input_stream: "feature_tensor" output_stream: "lapped_feature_tensor" input_side_packet: "BUFFER_SIZE:segment_size_int" input_side_packet: "OVERLAP:overlap_int" node_options: { [type.googleapis.com/mediapipe.LappedTensorBufferCalculatorOptions] { add_batch_dim_to_tensors: true } } }

node { calculator: "SidePacketToStreamCalculator" input_side_packet: "segment_size_int" output_stream: "AT_ZERO:segment_size_int_stream" }

node { calculator: "VectorIntToTensorCalculator" input_stream: "SINGLE_INT:segment_size_int_stream" output_stream: "TENSOR_OUT:segment_size_tensor" }

node { calculator: "PacketClonerCalculator" input_stream: "segment_size_tensor" input_stream: "lapped_feature_tensor" output_stream: "synced_segment_size_tensor" }

node { calculator: "TensorFlowSessionFromSavedModelCalculator" output_side_packet: "SESSION:session" node_options: {

  saved_model_path: "/tmp/mediapipe/saved_model"
}

} }

node: { calculator: "TensorFlowInferenceCalculator" input_side_packet: "SESSION:session" input_stream: "NUM_FRAMES:synced_segment_size_tensor" input_stream: "RGB_AND_AUDIO:lapped_feature_tensor" output_stream: "PREDICTIONS:prediction_tensor" node_options: {

  batch_size: 32
}

} }

node { calculator: "TensorToVectorFloatCalculator" input_stream: "prediction_tensor" output_stream: "prediction_vector" }

node { calculator: "TopKScoresCalculator" input_stream: "SCORES:prediction_vector" output_stream: "TOP_K_INDEXES:top_k_indexes" output_stream: "TOP_K_SCORES:top_k_scores" output_stream: "TOP_K_LABELS:top_k_labels" node_options: {

  top_k: 3
  label_map_path: "mediapipe/graphs/youtube8m/label_map.txt"
}

} }

node { calculator: "OpenCvVideoDecoderCalculator" input_side_packet: "INPUT_FILE_PATH:input_video_path" output_stream: "VIDEO:input_video" output_stream: "VIDEO_PRESTREAM:input_video_header" }

node { calculator: "LabelsToRenderDataCalculator" input_stream: "LABELS:top_k_labels" input_stream: "SCORES:top_k_scores" input_stream: "VIDEO_PRESTREAM:input_video_header" output_stream: "RENDER_DATA:render_data" node_options: {

  color { r: 255 g: 0 b: 0 }
  color { r: 0 g: 255 b: 0 }
  color { r: 0 g: 0 b: 255 }
  thickness: 2.0
  font_height_px: 20
  max_num_labels: 3
  location: TOP_LEFT
}

} }

node { calculator: "PacketClonerCalculator" input_stream: "render_data" input_stream: "input_video" output_stream: "synchronized_render_data" }

node { calculator: "AnnotationOverlayCalculator" input_stream: "INPUT_FRAME:input_video" input_stream: "synchronized_render_data" output_stream: "OUTPUT_FRAME:output_video" }

node { calculator: "OpenCvVideoEncoderCalculator" input_stream: "VIDEO:output_video" input_stream: "VIDEO_PRESTREAM:input_video_header" input_side_packet: "OUTPUT_FILE_PATH:output_video_path" node_options: {

  codec: "avc1"
  video_format: "mp4"
}

} }

I1204 00:53:43.325479 14636 simple_run_graph_main.cc:120] Initialize the calculator graph. I1204 00:53:43.326943 14636 simple_run_graph_main.cc:133] Start running the calculator graph. I1204 00:53:43.327373 14639 unpack_media_sequence_calculator.cc:203] Found feature timestamps: RGB/feature/timestamp with size: 7 I1204 00:53:43.327397 14639 unpack_media_sequence_calculator.cc:203] Found feature timestamps: AUDIO/feature/timestamp with size: 7 2019-12-04 00:53:43.327502: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /tmp/mediapipe/saved_model 2019-12-04 00:53:43.334811: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve } 2019-12-04 00:53:43.334861: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:264] Reading SavedModel debug info (if present) from: /tmp/mediapipe/saved_model 2019-12-04 00:53:43.334939: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE3 SSE4.1 SSE4.2 AVX AVX2 2019-12-04 00:53:43.371681: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:203] Restoring SavedModel bundle. 2019-12-04 00:53:43.417670: W external/org_tensorflow/tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 79093760 exceeds 10% of system memory. 2019-12-04 00:53:43.428570: W external/org_tensorflow/tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 94912512 exceeds 10% of system memory. 2019-12-04 00:53:43.523364: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:333] SavedModel load for tags { serve }; Status: success: OK. Took 195871 microseconds. [libx264 @ 0x7f8400f5fcc0] broken ffmpeg default settings detected [libx264 @ 0x7f8400f5fcc0] use an encoding preset (e.g. -vpre medium) [libx264 @ 0x7f8400f5fcc0] preset usage: -vpre -vpre [libx264 @ 0x7f8400f5fcc0] speed presets are listed in x264 --help [libx264 @ 0x7f8400f5fcc0] profile is optional; x264 defaults to high Could not open codec 'libx264': Unspecified errorW1204 00:53:44.755503 14636 opencv_video_decoder_calculator.cc:174] Not all the frames are decoded (total frames: 237 vs decoded frames: 29). E1204 00:53:44.776046 14636 simple_run_graph_main.cc:145] Failed to run the graph: CalculatorGraph::Run() failed in Run: Calculator::Process() for node "[OpenCvVideoEncoderCalculator, OpenCvVideoEncoderCalculator with node ID: 19 and input streams: <output_video,input_video_header>]" failed: ; Fail to open file at /tmp/mediapipe/annotated_video.mp4 `

jiuqiant commented 4 years ago

The error message is Could not open codec 'libx264': Unspecified errorW1204 00:53:44.755503 14636. It seems to be a ffmpeg/codec issue. I saw a similar issue with OpenCV2 before. Let's output the annotated video in avi format instead of mp4. You need to do the following two steps:

szyszy315 commented 4 years ago

@jiuqiant hello, it works! Thanks a lot. Now it can successfully generate annotated_video.avi. But i still don't know how to generate features.pb without any error. And I am trying to generate a tfrecord as the input of yt8m machine learning model, can you tell me how can i get the tfrecord? Thank you.

jiuqiant commented 4 years ago

It seems like features.pb is successfully generated but file I/O returns some error. I think you can modify the code here to be the following and print out the error indicator. Then, we can figure out what's going wrong.

return ::mediapipe::InternalErrorBuilder(MEDIAPIPE_LOC) 
    << "Error while writing file: " << file_name << " with error " << ferror(fp);
szyszy315 commented 4 years ago

@jiuqiant hello, i did edit the code but the error is the same as before, here is the error E1204 15:03:24.650466 11125 extract_yt8m_features.cc:130] Failed to run the graph: ; Error while writing file: /tmp/mediapipe/features.pb

jiuqiant commented 4 years ago

Do you rebuild the binary?

szyszy315 commented 4 years ago

@jiuqiant hello, i rebuilded the binary but the error doesn't change.

jiuqiant commented 4 years ago

The error message still doesn't have the error indicator after the rebuild?

szyszy315 commented 4 years ago

@jiuqiant Yes. I modified the code and ran bazel build -c opt \ --define MEDIAPIPE_DISABLE_GPU=1 --define no_aws_support=true \ mediapipe/examples/desktop/youtube8m:extract_yt8m_features and GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/youtube8m/extract_yt8m_features \ --calculator_graph_config_file=mediapipe/graphs/youtube8m/feature_extraction.pbtxt \ --input_side_packets=input_sequence_example=/tmp/mediapipe/metadata.pb \ --output_side_packets=output_sequence_example=/tmp/mediapipe/features.pb and didn't get error indicator.

szyszy315 commented 4 years ago

@jiuqiant and when i do inference with this tfrecord file the output is always an empty csv file python \ inference.py --train_dir ~/yt8m/models/frame/sample_model \ --output_file=$HOME/tmp/kaggle_solution.csv \ --input_data_pattern=${HOME}/yt8m/3/frame/test/test*.tfrecord --segment_labels

jiuqiant commented 4 years ago

@jiuqiant and when i do inference with this tfrecord file the output is always an empty csv file python \ inference.py --train_dir ~/yt8m/models/frame/sample_model \ --output_file=$HOME/tmp/kaggle_solution.csv \ --input_data_pattern=${HOME}/yt8m/3/frame/test/test*.tfrecord --segment_labels

First, features.pb is a serialized sequence example, if you want to wrap it as a tfrecord, you need to write some python code to do so. Secondly, features.pb can't be fed to the model directly. That's why we made the local_video_model_inference.pbtxt. If you plan to use it with inference.py, you need to follow the logic in the local_video_model_inference.pbtxt to make an input tensor with name RGB_AND_AUDIO.

szyszy315 commented 4 years ago

@jiuqiant Thank you for your reply. I get the error indicator now: E1209 17:34:52.621446 21112 extract_yt8m_features.cc:130] Failed to run the graph: ; Error while writing file: /tmp/mediapipe/features.pb with error 1 and can you talk more about what i should do now? I'm a newbie and can't understand what 'make an input tensor with name RGB_AND_AUDIO' means. Thank you.

Magsun commented 4 years ago

@jiuqiant the size of feature.pb is 32.9 kB (32,882 bytes), and i can successfully run the script here is a part of the content of feature.pb. feature { float_list { value: 2.076115369796753 value: 1.1078059673309326 value: -0.18813468515872955 value: -0.8384913206100464 value: -0.11542361229658127 value: -0.2440829873085022 value: -1.3306479454040527 value: 0.7086437344551086 value: 1.4460822343826294 value: -0.884070098400116 value: 0.4248296618461609 value: -0.13979442417621613 value: 0.9824615120887756 value: 0.5154528617858887 value: -1.5065950155258179 value: -0.73182612657547 value: -0.9751976728439331 value: 1.3784875869750977 value: 0.44272834062576294 value: 0.5880029797554016 value: -1.0954900979995728 value: 0.18842922151088715 value: -0.283181756734848 value: 1.825642704963684 value: 1.159321665763855 value: 0.584626317024231 value: 2.486922025680542 value: 0.1595749855041504 value: -0.004658078774809837 value: 1.1477636098861694 value: 1.0637904405593872 value: 1.108216643333435 value: -1.7783350944519043 value: -0.9082486033439636 value: -0.7650951147079468 value: -1.308105230331421 value: 1.6427212953567505 value: -0.15743005275726318 value: -0.03219599276781082 value: 1.384970784187317 value: -1.3257311582565308 value: 0.030308924615383148 value: -2.8624725341796875 value: 0.17116490006446838 value: -0.22443348169326782 value: -1.5767983198165894 value: -1.429734706878662 value: 1.3550169467926025 value: 0.015885576605796814 value: 0.0877981185913086 value: -0.06826288253068924 value: -0.7793705463409424 value: 2.0656070709228516 value: -0.516409695148468 value: 0.8668118119239807 value: -0.041077546775341034 value: -1.3959709405899048 value: -0.08360891044139862 value: -1.9882359504699707 value: 0.949579119682312 value: -2.0266823768615723 value: -1.0518616437911987 value: 1.216860055923462 value: -1.0414947271347046 value: 2.06475830078125 value: -1.900275468826294 value: -0.2767667770385742 value: -1.614473581314087 value: -0.925095796585083 value: -3.356851577758789 value: 0.8574276566505432 value: 0.9824469089508057 value: 1.5532904863357544 value: -1.303615927696228 value: 1.1339962482452393 value: 1.007266879081726 value: 0.1647041141986847 value: -0.6073731780052185 value: -1.9708610773086548 value: -0.709756076335907 value: -2.327376127243042 value: -0.011305160820484161 value: 0.0936271995306015 value: 0.7555102705955505 value: -0.09514406323432922 value: -1.7542684078216553 value: 2.4782512187957764 value: 1.2873051166534424 value: -0.4111500084400177 value: -1.6801884174346924 value: -0.5894775390625 value: -0.42311108112335205 value: 0.1963639259338379 value: -0.04439619556069374 value: 2.3887453079223633 value: 2.415273904800415 value: -0.7920663952827454 value: -1.2967287302017212 value: -1.0240007638931274 value: 1.57100248336792 value: -1.5513333082199097 value: -3.261763572692871 value: -0.8368674516677856 value: 2.48105788230896 value: -0.8066669702529907 value: -1.6707193851470947 value: -0.11786305904388428 value: 0.5136384963989258 value: 0.926856517791748 value: -0.847081184387207 value: -1.1720112562179565 value: 0.8009216785430908 value: -1.550827980041504 value: -0.9219430088996887 value: -0.7456883192062378 value: 0.8307655453681946 value: 1.8678431510925293 value: 3.582793951034546 value: -0.4868828356266022 value: 2.4303548336029053 value: 0.08972973376512527 value: -1.8924450874328613 value: -1.6072789430618286 value: 0.7497991323471069 value: 0.031924206763505936 value: 0.612067699432373 value: -0.7399752140045166 value: 0.31601041555404663 value: -0.6428033709526062 value: -3.2113521099090576 value: 0.31452426314353943 value: -3.185826063156128 value: -2.122425079345703 value: 1.3172622919082642 value: -0.011581912636756897 value: -0.4665193259716034 value: -0.575229823589325 value: -0.44019705057144165 value: 0.752683162689209 value: 0.09542949497699738 value: -1.8906596899032593 value: -1.0785622596740723 value: 1.2238408327102661 value: -0.4560732841491699 value: 0.943932294845581 value: -0.09849487990140915 value: 1.3175866603851318 value: 1.0762571096420288 value: 0.9753843545913696 value: 2.137068271636963 value: 0.3518795669078827 value: -0.008807189762592316 value: 0.24545150995254517 value: -1.4994237422943115 value: -1.3719347715377808 value: 0.7721972465515137 value: -0.8061539530754089 value: 1.6750097274780273 value: -1.6375633478164673 value: -0.08622197806835175 value: 1.820083737373352 value: 3.42944073677063 value: 1.0887162685394287 value: 1.6489651203155518 value: -0.4037898778915405 value: -0.2327711582183838 value: -1.2390763759613037 value: 0.4883323013782501 value: -0.6950839161872864 value: 1.7811808586120605 value: 1.7405081987380981 value: -0.23283472657203674 value: 0.24460241198539734 value: -1.1376416683197021 value: 0.9089964032173157 value: 3.5345003604888916 value: 1.2678627967834473 value: 1.9292559623718262 value: 0.8725306987762451 value: -1.0869665145874023 value: 0.5660413503646851 value: 0.13393375277519226 value: -0.24879832565784454 value: 0.6325265765190125 value: 0.09257601946592331 value: 1.9454909563064575 value: 0.8988593816757202 value: 0.22737044095993042 value: 0.3642851710319519 value: -0.4734339714050293 value: -2.0749330520629883 value: -0.47319236397743225 value: 0.09216434508562088 value: 1.658652424812317 value: 0.6806251406669617 value: -0.5392337441444397 value: 1.648949384689331 value: 1.4028279781341553 value: -0.19719460606575012 value: 0.14736446738243103 value: 1.493246078491211 value: 1.3111799955368042 value: -0.5843704342842102 value: -0.38041794300079346 value: 2.3881468772888184 value: -0.5753710269927979 value: 0.5786773562431335 value: -1.5756491422653198 value: -1.5164716243743896 value: -0.7557620406150818 value: -0.07720915973186493 value: -0.14232070744037628 value: -0.3186644911766052 value: -0.26369768381118774 value: -1.6858769655227661 value: -0.02637369930744171 value: -0.2754836976528168 value: 0.4108368456363678 value: 0.13638675212860107 value: 1.848016619682312 value: 1.7002733945846558 value: -2.4318244457244873 value: 2.9304964542388916 value: 1.212127685546875 value: 0.3838869631290436 value: -0.8915154933929443 value: -1.9818795919418335 value: 0.8447856903076172 value: -0.4125208854675293 value: -0.8733507394790649 value: -0.986823558807373 value: -0.21005408465862274 value: -0.9300406575202942 value: -1.628851294517517 value: 0.05673721060156822 value: 1.2767287492752075 value: -1.8176496028900146 value: -2.134795904159546 value: -0.5140668749809265 value: 0.27915555238723755 value: 0.26706749200820923 value: 0.9186589121818542 value: 1.3928263187408447 value: -0.3353036642074585 value: 4.319987773895264 value: 1.0011574029922485 value: -0.1646348237991333 value: 1.5651878118515015 value: -0.13341094553470612 value: -0.9453008770942688 value: -1.1091512441635132 value: 1.2046048641204834 value: -1.8501906394958496 value: 1.0726304054260254 value: 1.1842310428619385 value: -0.024764731526374817 value: -0.7953068614006042 value: 0.6875433921813965 value: 0.010663069784641266 value: -0.20165225863456726 value: 0.39678940176963806 value: -0.6475030183792114 value: -1.878673791885376 value: -0.079218290746212 value: -1.8984785079956055 value: -0.36981743574142456 value: -1.1236318349838257 value: 1.4404150247573853 value: -1.3435301780700684 value: 0.33469414710998535 value: 0.08946636319160461 value: 0.8926478624343872 value: -0.8621282577514648 value: -0.5364977121353149 value: -0.5715200304985046 value: -0.0524345338344574 value: 0.5212527513504028 value: -0.036801666021347046 value: 0.7975624203681946 value: 2.055530071258545 value: -0.7584375143051147 value: 0.07001752406358719 value: -3.543323516845703 value: -0.23199038207530975 value: -2.6343026161193848 value: -1.4576836824417114 value: -1.8611493110656738 value: -0.2536201477050781 value: 2.48909592628479 value: -0.04734961315989494 value: 0.5331287384033203 value: -0.671368420124054 value: -0.0707893818616867 value: -2.6089415550231934 value: -0.19841080904006958 value: -1.468470573425293 value: 0.05417454242706299 value: -2.8305346965789795 value: -0.061848171055316925 value: -1.006321907043457 value: 0.25219088792800903 value: 1.4386223554611206 value: 0.579887330532074 value: -0.4711796045303345 value: 0.02917114645242691 value: 1.1335608959197998 value: 1.1308274269104004 value: 2.3994791507720947 value: 2.2572693824768066 value: -1.7117764949798584 value: 1.9838805198669434 value: 2.0533621311187744 value: -0.7716501355171204 value: 0.2622033357620239 value: 0.6035615801811218 value: 2.0593576431274414 value: -0.1605762243270874 value: -1.2871464490890503 value: -2.9039580821990967 value: 2.850698947906494 value: -1.27129328250885 value: -0.42772600054740906 value: -0.8661221265792847 value: -2.9051530361175537 value: 0.8335804343223572 value: 0.9288882613182068 value: -0.6807757019996643 value: 2.3977253437042236 value: -0.5715345144271851 value: -1.8101439476013184 value: -0.629018247127533 value: -0.28434693813323975 value: -1.5168163776397705 value: 0.34159472584724426 value: 0.2774682641029358 value: -0.3267296254634857 value: -0.7374221682548523 value: 1.4254999160766602 value: 0.06263265013694763 value: -2.6425418853759766 value: -1.8985848426818848 value: 1.5762429237365723 value: 0.6307982802391052 value: -1.3759032487869263 value: -0.490688681602478 value: 0.22378122806549072 value: -0.693402111530304 value: -1.695326805114746 value: -2.150895833969116 value: -0.7059016823768616 value: -1.5648071765899658 value: -1.7223933935165405 value: 1.099951982498169 value: 0.7067866921424866 value: -0.5848215222358704 value: -1.5359903573989868 value: 2.0208513736724854 value: 1.3732399940490723 value: 0.9065101146697998 value: -0.3077787160873413 value: -0.4899407923221588 value: 2.199974298477173 value: -0.21433964371681213 value: -1.5456434488296509 value: 1.6424561738967896 value: -1.286812663078308 value: 0.4654316008090973 value: 1.2778654098510742 value: 0.6953151226043701 value: -1.4307490587234497 value: 1.3815690279006958 value: -2.9147303104400635 value: -0.280474454164505 value: 0.00995706021785736 value: -0.44313567876815796 value: 1.01633882522583 value: 0.9478065371513367 value: 0.3741763234138489 value: -0.12752550840377808 value: 1.0716766119003296 value: 0.36232084035873413 value: 0.4079861044883728 value: 0.7223873734474182 value: 0.34057605266571045 value: 0.5880310535430908 value: 0.7112571597099304 value: 0.12893719971179962 value: -2.4549248218536377 value: -0.8952239155769348 value: 1.4323134422302246 value: 1.0494647026062012 value: -1.498865008354187 value: -1.9658281803131104 value: 0.20043067634105682 value: -3.62387752532959 value: -0.8979582786560059 value: -1.5463080406188965 value: -0.8505302667617798 value: -0.5094913244247437 value: 2.2940118312835693 value: -0.9385546445846558 value: 1.328762412071228 value: -0.7265905141830444 value: -0.2145901322364807 value: -0.04741540551185608 value: -0.1488470733165741 value: 1.7708771228790283 value: 1.8799668550491333 value: 1.301935076713562 value: -0.6532919406890869 value: 1.0260339975357056 value: 0.6983137130737305 value: -0.9304280281066895 value: 1.076409101486206 value: -0.9686469435691833 value: -2.054157018661499 value: -0.8036925792694092 value: -1.4463199377059937 value: -0.5630106925964355 value: -0.10041627287864685 value: 0.2565438449382782 value: -2.1981208324432373 value: -1.1467270851135254 value: -0.7692071795463562 value: 1.005334734916687 value: 1.5203325748443604 value: -0.9795114994049072 value: -1.3341177701950073 value: -0.6978042125701904 value: -1.3946025371551514 value: -2.3340437412261963 value: -0.5995053052902222 value: -0.6093590259552002 value: -1.1362119913101196 value: 0.965411901473999 value: -0.5325666666030884 value: 1.5325769186019897 value: -1.2943822145462036 value: -0.10280214250087738 value: 0.851248025894165 value: -1.9097346067428589 value: -0.11335210502147675 value: 1.5964782238006592 value: -0.045951440930366516 value: -0.5869671106338501 value: -2.526352882385254 value: -0.5252227783203125 value: 3.331969738006592 value: -0.328927218914032 value: -0.31322500109672546 value: -0.41437578201293945 value: 1.204060435295105 value: -1.0777133703231812 value: 1.507775902748108 value: -0.3260658085346222 value: 1.928621530532837 value: -1.2279019355773926 value: -0.3879219889640808 value: 0.5551758408546448 value: 1.8628013134002686 value: 2.050586223602295 value: -3.7830770015716553 value: 1.4697057008743286 value: 0.3545951843261719 value: -1.0896382331848145 value: 0.49237409234046936 value: -0.9073858261108398 value: -1.1787737607955933 value: -0.5141935348510742 value: 0.10295286774635315 value: 2.2614545822143555 value: 0.5850622057914734 value: -0.48616135120391846 value: -2.161655902862549 value: -0.45164385437965393 value: 1.1072373390197754 value: -0.86528080701828 value: -2.2031350135803223 value: -0.43436387181282043 value: 0.4707050621509552 value: 1.759995460510254 value: 1.1927800178527832 value: -0.5998307466506958 value: -0.4844651520252228 value: 0.08951327204704285 value: -1.231548547744751 value: 1.0700745582580566 value: -0.7404585480690002 value: -0.5306257009506226 value: -1.053593635559082 value: -1.4943562746047974 value: 0.33376118540763855 value: 0.24344927072525024 value: 0.3685351014137268 value: -0.5092860460281372 value: -0.611930251121521 value: 0.7611799240112305 value: -0.31345120072364807 value: -1.129486322402954 value: -1.9233026504516602 value: 2.781923294067383 value: 0.11787816882133484 value: -0.15942081809043884 value: -1.5138243436813354 value: 0.17782726883888245 value: -1.6063125133514404 value: -0.9038397073745728 value: 0.9823489189147949 value: 2.758408308029175 value: -0.01957005262374878 value: 1.5423431396484375 value: 2.3222010135650635 value: -0.6891860365867615 value: -0.9210190176963806 value: 1.7100516557693481 value: 1.5623365640640259 value: -1.707276701927185 value: 0.6837890148162842 value: -1.0777784585952759 value: -1.6204429864883423 value: -2.059587001800537 value: -0.14859679341316223 value: 0.20182068645954132 value: -3.0875930786132812 value: 1.1198688745498657 value: 1.2911971807479858 value: 0.14077423512935638 value: -1.421191930770874 value: 0.779792845249176 value: 0.859072208404541 value: 3.994802951812744 value: 0.1280965656042099 value: -1.0795221328735352 value: 0.6918175220489502 value: -0.7177660465240479 value: -1.715369701385498 value: -0.23453421890735626 value: 0.36311763525009155 value: 0.5149015188217163 value: 0.8318600654602051 value: 0.22643527388572693 value: 0.13885849714279175 value: -1.276604175567627 value: 2.4378278255462646 value: -0.17550931870937347 value: 0.7171204090118408 value: 0.00713212788105011 value: 0.8381497859954834 value: -0.6088944673538208 value: 0.4728585481643677 value: -0.04492643475532532 value: -0.26995864510536194 value: -1.9193251132965088 value: -3.3169825077056885 value: 0.8015750646591187 value: 0.408490389585495 value: 0.09775852411985397 value: 0.132622629404068 value: -1.9742239713668823 value: -0.33567988872528076 value: 1.2375240325927734 value: -2.184105157852173 value: 0.5942186117172241 value: -0.4967365264892578 value: 0.0779062807559967 value: 1.1705176830291748 value: -0.6791409850120544 value: 1.4983488321304321 value: -0.237945094704628 value: -0.6321359872817993 value: -1.5501048564910889 value: -0.7882075309753418 value: -0.8763176798820496 value: -1.3156052827835083 value: -1.0282502174377441 value: -1.1174026727676392 value: 2.293895959854126 value: -2.11755108833313 value: 0.40874210000038147 value: 2.167431116104126 value: 0.9265503883361816 value: -0.1037089079618454 value: -1.3828001022338867 value: -1.254130244255066 value: -2.329331398010254 value: 1.2428333759307861 value: -0.5488374829292297 value: 0.36707204580307007 value: 1.1477612257003784 value: -0.26736360788345337 value: 0.7133390307426453 value: 1.6296908855438232 value: -1.4705464839935303 value: -0.5046520233154297 value: -0.7684431076049805 value: 0.28728359937667847 value: -1.5011811256408691 value: -0.079474076628685 value: -1.017632246017456 value: 2.0540554523468018 value: -1.6730924844741821 value: 0.060628294944763184 value: -0.061575233936309814 value: -1.0044909715652466 value: 3.257035732269287 value: 1.7926173210144043 value: -2.1903316974639893 value: -0.10165868699550629 value: 0.32420504093170166 value: 0.4139823615550995 value: 0.5033915638923645 value: 1.6441519260406494 value: -0.9672199487686157 value: -2.2941830158233643 value: -1.3591971397399902 value: -0.41222211718559265 value: -1.1566503047943115 value: 1.4126023054122925 value: 0.16003163158893585 value: 1.7526302337646484 value: 0.7163461446762085 value: 0.03353457897901535 value: 0.7607404589653015 value: 1.9751901626586914 value: -1.4662978649139404 value: 0.32588014006614685 value: -2.507495880126953 value: -0.05304610729217529 value: -1.522810935974121 value: 0.43970105051994324 value: 2.196805477142334 value: -1.9686163663864136 value: -0.17659585177898407 value: 1.446294903755188 value: -1.0893762111663818 value: 0.582170844078064 value: -1.2488877773284912 value: -0.9753806591033936 value: 1.6396675109863281 value: -1.1663167476654053 value: 0.5754790306091309 value: 0.5257911682128906 value: 0.1384410858154297 value: -2.1029715538024902 value: -0.07679685950279236 value: -0.07024064660072327 value: 4.212103366851807 value: -0.6996765732765198 value: 0.49270692467689514 value: -0.8458765149116516 value: 1.3353288173675537 value: 0.5400585532188416 value: 0.3836984634399414 value: 1.8173166513442993 value: -1.663107991218567 value: -1.7252086400985718 value: -1.0992074012756348 value: 1.3622026443481445 value: 0.2867245078086853 value: -0.04090416431427002 value: 0.4595750868320465 value: 0.7557359933853149 value: 0.6710014939308167 value: -2.4956390857696533 value: 0.6815018653869629 value: 1.5328012704849243 value: 1.0107685327529907 value: -1.0203531980514526 value: 0.6467156410217285 value: -0.4034999907016754 value: 2.6777799129486084 value: 0.4509180188179016 value: 1.3818395137786865 value: 1.9401068687438965 value: 0.8671821355819702 value: 0.09059834480285645 value: 2.3978803157806396 value: 0.5636197328567505 value: -1.1829209327697754 value: 0.9495004415512085 value: 0.1274394989013672 value: -1.750793218612671 value: 0.6026861667633057 value: -0.1557961106300354 value: 1.6593365669250488 value: -0.7877292633056641 value: 0.9259800910949707 value: 0.22194582223892212 value: 0.33618858456611633 value: 0.7520437240600586 value: -0.9863332509994507 value: 0.43588465452194214 value: -0.12301796674728394 value: 0.39134934544563293 value: -0.6367123126983643 value: 0.1919347643852234 value: -1.0463886260986328 value: -1.2894504070281982 value: -1.5288201570510864 value: 1.658541202545166 value: 0.5257874727249146 value: 0.5543509721755981 value: 0.8607103228569031 value: 0.951469361782074 value: 1.5920846462249756 value: -1.4318946599960327 value: 0.47628602385520935 value: -0.6824831962585449 value: 1.4573286771774292 value: 0.3038623332977295 value: -0.06113210320472717 value: 1.058286428451538 value: -0.08354541659355164 value: -0.5825676321983337 value: -0.08145475387573242 value: 0.2608497142791748 value: 1.242293119430542 value: -0.32986271381378174 value: -0.41835957765579224 value: 1.2640823125839233 value: -0.08283620327711105 value: -0.16441693902015686 value: 1.156849980354309 value: -1.387986660003662 value: -1.4179691076278687 value: 0.8726122379302979 value: -2.169668436050415 value: -0.7791728377342224 value: -1.4190475940704346 value: 0.188334658741951 value: -0.31121960282325745 value: 0.8686013221740723 value: 0.12810254096984863 value: 3.1133041381835938 value: -0.6583575010299683 value: -1.891268014907837 value: -1.71553635597229 value: -1.2640318870544434 value: -2.125417470932007 value: -2.2403831481933594 value: -1.3303781747817993 value: 0.10707536339759827 value: -0.10819298028945923 value: 0.7587528228759766 value: -3.6602096557617188 value: -0.10786467790603638 value: -0.8920912146568298 value: 1.5344736576080322 value: 0.4542205333709717 value: 0.2542000412940979 value: -0.9001966714859009 value: -0.850999653339386 value: 0.2876558005809784 value: 0.5929479598999023 value: 0.2646709084510803 value: 0.8582943677902222 value: 1.227980136871338 value: 0.1025797426700592 value: 0.25967758893966675 value: -0.6584393978118896 value: 1.8413779735565186 value: 0.018914014101028442 value: -0.8523246049880981 value: 1.05805242061615 value: 0.08580684661865234 value: -1.0977952480316162 value: 0.41021043062210083 value: -0.2414032220840454 value: -0.8463214039802551 value: -2.2171831130981445 value: -2.2572672367095947 value: -1.536203384399414 value: 0.662703275680542 value: 2.250798225402832 value: -1.529068112373352 value: 1.1428463459014893 value: -2.21043062210083 value: -0.17448699474334717 value: -0.621275007724762 value: -1.462862253189087 value: 0.35131120681762695 value: 0.17737379670143127 value: -0.4714112877845764 value: -1.0097854137420654 value: 0.08917000889778137 value: 0.6742238402366638 value: 0.9743010997772217 value: -1.0466209650039673 value: -1.05338454246521 value: -1.4871054887771606 value: -0.6949794292449951 value: -0.9471079707145691 value: -1.2720668315887451 value: 0.23235979676246643 value: -0.08154436945915222 value: 1.1909196376800537 value: -0.3748297095298767 value: -0.8733246922492981 value: 1.2948898077011108 value: -0.24685853719711304 value: -0.3131599724292755 value: 0.4706265330314636 value: 1.5272151231765747 value: 1.5599596500396729 value: -1.4874463081359863 value: -1.3331787586212158 value: -1.7727274894714355 value: -0.08877497911453247 value: -1.6576133966445923 value: 0.11239799857139587 value: -0.13919253647327423 value: 0.6881515383720398 value: -0.9482897520065308 value: -0.3039206266403198 value: -0.20627111196517944 value: -1.5109062194824219 value: 0.24225211143493652 value: -0.009042799472808838 value: -2.4743900299072266 value: -1.6658531427383423 value: -0.32486337423324585 value: -0.8449218273162842 value: 1.2172571420669556 value: 1.905389666557312 value: -1.2685644626617432 value: -2.5383994579315186 value: 0.9311712384223938 value: 0.5448826551437378 value: 0.15401095151901245 value: 1.1081100702285767 value: -0.5396357178688049 value: 1.155993938446045 value: 1.2488160133361816 value: 0.8588473200798035 value: 0.44656795263290405 value: 1.171327829360962 value: -0.2690891623497009 value: 1.3756144046783447 value: 0.7742607593536377 value: 2.5948703289031982 value: -0.8630634546279907 value: 0.2800670266151428 value: 1.4807395935058594 value: -1.5012378692626953 value: 0.3438887596130371 value: 0.7298605442047119 value: 0.6276078224182129 value: -1.3623075485229492 value: -1.0935521125793457 value: -1.0911192893981934 value: 0.4304130971431732 value: 0.09266266226768494 value: -1.357710838317871 value: -0.5553179979324341 value: -0.28151288628578186 value: -1.9078547954559326 value: -0.41531217098236084 value: -1.1181950569152832 value: 0.8765799403190613 value: -1.0592482089996338 value: -1.9457340240478516 value: 0.8603631258010864 value: 1.6130988597869873 value: 0.31300288438796997 value: 1.5846459865570068 value: 0.10774415731430054 value: -0.05864184349775314 value: 1.5854884386062622 value: -0.7500016093254089 value: -0.7805182933807373 value: -1.533337116241455 value: -0.19407367706298828 value: -0.5015048980712891 value: -1.0564229488372803 value: 0.7987643480300903 value: 0.6627199053764343 value: -0.03455421328544617 value: -2.043048143386841 value: 0.33498305082321167 value: -0.12174426019191742 value: 2.8576321601867676 value: 0.6044117212295532 value: -2.24442458152771 value: -1.1007792949676514 value: -1.6784660816192627 value: 1.3082897663116455 value: 0.36610108613967896 value: -0.7117820382118225 value: 1.445816993713379 value: 1.212733268737793 value: 0.20808029174804688 value: 0.805008590221405 value: -0.4894181489944458 value: -0.50527024269104 value: -1.087148666381836 value: 1.7483556270599365 value: -1.3845714330673218 value: -0.027503877878189087 value: 0.656522274017334 value: 0.07468867301940918 value: 0.8858073949813843 value: -0.31627973914146423 value: 1.1880061626434326 value: -1.6633497476577759 value: -2.947570323944092 value: -2.1041316986083984 value: 0.065956711769104 value: 1.238690972328186 value: -0.670972466468811 value: 0.4631921052932739 value: -0.22584708034992218 value: -0.014074534177780151 value: 1.2271747589111328 value: -0.1408875286579132 value: -1.0871182680130005 value: -0.37171247601509094 value: 0.3868141174316406 value: -1.3457255363464355 value: -0.141131192445755 value: 0.4967617690563202 value: 0.1685529351234436 value: -1.6076552867889404 value: 1.547536849975586 value: -1.2592315673828125 value: 2.341219425201416 value: -0.19430238008499146 value: 0.6378706097602844 value: -0.5116959810256958 value: -2.6581196784973145 value: -0.3645663261413574 value: -0.749751091003418 value: 0.6051032543182373 value: 1.4998607635498047 value: -1.2030404806137085 value: -0.24497655034065247 value: 0.6152099370956421 value: 0.7808939814567566 value: -1.1130836009979248 value: 0.7788217067718506 value: 0.013284295797348022 value: 0.6142373085021973 value: 1.536675214767456 value: 0.6126294732093811 value: 1.4795689582824707 value: -2.3028745651245117 value: -1.0344455242156982 value: 0.6834166049957275 value: -0.2190404236316681 value: 0.30404895544052124 value: -0.5494284629821777 value: -1.1515576839447021 value: 0.8520990014076233 value: -1.2436439990997314 value: -0.2021297961473465 value: 2.032911539077759 value: -0.20717772841453552 value: 2.3595993518829346 value: 0.042632028460502625 value: 1.0470619201660156 value: 1.260422945022583 value: -0.7738087773323059 value: 2.130340337753296 value: 1.127643346786499 value: -0.22513914108276367 value: 0.545540452003479 value: -0.4573346972465515 value: -0.2560608983039856 value: -3.6818153858184814 value: -0.22341525554656982 value: -0.26421135663986206 value: -3.1989917755126953 value: -0.12157242000102997 value: -1.0877983570098877 value: 2.9311070442199707 value: 1.130705714225769 value: -0.7277622222900391 value: -1.2124369144439697 value: -1.4359900951385498 value: 1.0710351467132568 value: -0.4723031520843506 value: -0.022355198860168457 value: -0.655505359172821 value: 1.9591941833496094 value: 2.8833367824554443 value: 0.8208673596382141 value: -0.6619185209274292 value: 0.368146151304245 value: -0.19281776249408722 value: 0.21288150548934937 value: 3.2281436920166016 value: -2.129258632659912 value: 2.809626817703247 value: 0.8000625371932983 value: -0.1811503767967224 value: -0.4792667031288147 value: -1.347848892211914 value: 0.28895899653434753 value: 1.5758254528045654 value: 0.09923964738845825 value: -0.5243102312088013 value: -0.5460996627807617 value: 0.6530378460884094 value: -0.13622522354125977 value: -0.17016145586967468 value: 0.5283380746841431 value: -1.8515403270721436 value: -1.0958117246627808 value: 0.05082771182060242 value: 2.11309552192688 value: 2.02075457572937 value: -0.5967410802841187 value: -0.017846886068582535 value: -1.2482661008834839 value: -0.8981242179870605 value: -1.0454866886138916 value: 1.0482861995697021 value: -0.5114814043045044 value: -2.2299327850341797 value: 0.6608255505561829 value: -0.4123225510120392 value: 2.909165382385254 value: -0.553383469581604 value: -0.3688572645187378 value: 0.984059751033783 } } } } feature_list { key: "RGB/feature/timestamp" value { feature { int64_list { value: 0 } } feature { int64_list { value: 1000000 } } feature { int64_list { value: 2000000 } } feature { int64_list { value: 3000000 } } feature { int64_list { value: 4000000 } } feature { int64_list { value: 5000000 } } feature { int64_list { value: 6000000 } } } } }

hello I'm trying to use mediapipe to create yt8m style dataset from my own videos, as I check for yt8m data, the value of audio & rgb is byteslist instead of float_list. And I found that older version of feature extractor only extract rgb features from local video and quantize float to bytes, but for audio feature just insert zeros. I don't know if I should quantize mediapipe extracted features to bytes and how to do this. @jiuqiant Kindly help, thanks a lot.