tinyyolov2.pb file

[topilski]

Hey, where I can find tinyyolov2.pb , i want to test: tinyyolov2 + detectionoverlay.

[GFallasRR]

Hi @topilski, the models are available at our model zoo. Please follow this link: Model Zoo

We also have examples about how to use TinyYoloV2. Link to examples: Examples

Regards.

[topilski]

Hey, @GFallasRR thank you for the reply, what should contain this folder INPUT_LAYER='input/Placeholder'? I also generated .pb file from darknet models is it correct? Also, you posted broked links.

[GFallasRR]

Hi @topilski, The links are fixed.

INPUT_LAYER='input/Placeholder'

This is not a folder, it is the input node name. If you use a model from darknet, you must check the input and output node names to properly use the model.

Regards.

[topilski]

How can I generate labels.txt ?

[GFallasRR]

That file should be included in TinyYoloV2_TensorFlow.tar.gz

[topilski]

This file redirect to: https://shop.ridgerun.com/products/tinyyolov2-for-tensorflow , add to chart what i should do to receive tar.gz?

[GFallasRR]

The model is downloaded from the shop, you just need to provide some contact information before downloading the file.

[topilski]

Ok, I see, so in future, it will be paid?

[GFallasRR]

The pre-trained models provided in the model zoo are used as reference, you can download the model from the website at no cost, you need to log and fill information at our store to download models, but its price is $0.

We provide professional services to train specific models but this requires coordinating, indicating models requirements, discuss about training sets and others, for this type of work billable hours are required, we work with a "time and materials" model in those cases.

We also extend GstInference to manage new network architectures for this we provide the service as time and materials, if you require this service we can coordinate a call and discuss about your project requirements.

Please visit our store for more information: shop

[topilski]

@GFallasRR thank you for your answers.

[topilski]

Why you need tee element in this pipeline?

v4l2src device=$CAMERA ! "video/x-raw, width=1280, height=720" ! tee name=t \
t. ! videoconvert ! videoscale ! queue ! net.sink_model \
t. ! queue ! net.sink_bypass \
inceptionv4 name=net model-location=$MODEL_LOCATION backend=tensorflow backend::input-layer=$INPUT_LAYER  backend::output-layer=$OUTPUT_LAYER \
net.src_bypass ! videoconvert ! classificationoverlay labels="$(cat $LABELS)" font-scale=4 thickness=4 ! videoconvert ! xvimagesink sync=false

[GFallasRR]

The Inference plugins have 2 pads (model and bypass).

The model pad scales the input image according to the model input. The bypass pad doesn't modify the original image, the inference result from the model pad is copied to the image in the bypass pad.

What is the purpose of this structure? Let's use an example: Using a Full HD webcam as input, the image size is 1920x1080. InceptionV1 requires 224x224 so we need to scale the input image from 1920x1080 to 224x224, this scaled image is used in the model pad. We can use model pad output to do the overlay but that image will be scaled (224x224).

The tee element creates a copy of Full HD image to be used in the bypass pad. When the inference is done, we have the results in the model pad and bypass pad.

Finally, if we use the output of bypass pad with overlay, we will obtain Full HD images with valid inference results. This is the pipeline structure in your example.

[topilski]

@GFallasRR thank you for the reply, it is very interesting but not portable if you need 224x224 sizes on src you need to write this in element spec. For example, after postproc I want to integrate your element into an existing pipeline how can I connect it? I should create tee and work with pads by requests? in this case, better create bin element which will have tee and needed elements. Here part of code:

const auto aratio = conf->GetAspectRatio();
if (aratio) {
      elements::video::ElementAspectRatio* aspect_ratio =
          new elements::video::ElementAspectRatio(common::MemSPrintf(ASPECT_RATIO_NAME_1U, video_id));
      aspect_ratio->SetAspectRatio(*aratio);
      ElementAdd(aspect_ratio);
      ElementLink(last, aspect_ratio);
      last = aspect_ratio;
 }

  if (framerate) {
      last = elements::encoders::build_video_framerate(*framerate, this, last, video_id);
  }

  const auto deep_learning = conf->GetDeepLearning();
  if (deep_learning) {
    elements::deep_learning::ElementTinyYolov2* tiny =
        new elements::deep_learning::ElementTinyYolov2(common::MemSPrintf("tiny_%lu", video_id));
    tiny->SetBackend(deep_learning->GetBackend());
    tiny->SetModelLocation(deep_learning->GetModelPath().GetPath());
    for (auto prop : deep_learning->GetProperties()) {
      tiny->SetBackendProperty(prop.property, prop.value);
    }

    ElementAdd(tiny);
    ElementLink(last, tiny);
    last = tiny;
  }

  const auto logo = conf->GetLogo();
  if (logo) {
    common::uri::Url logo_uri = logo->GetPath();
    common::draw::Point logo_point = logo->GetPosition();
    alpha_t alpha = logo->GetAlpha();
    elements::video::ElementGDKPixBufOverlay* videologo =
        new elements::video::ElementGDKPixBufOverlay(common::MemSPrintf(VIDEO_LOGO_NAME_1U, video_id));
    common::uri::Url::scheme scheme = logo_uri.GetScheme();
    if (scheme == common::uri::Url::file) {
      common::uri::Upath upath = logo_uri.GetPath();
      std::string path = upath.GetPath();
      videologo->SetLocation(path);
    } else {
      NOTREACHED();
    }
    videologo->SetOffsetX(logo_point.x);
    videologo->SetOffsetY(logo_point.y);
    videologo->SetAlpha(alpha);
    ElementAdd(videologo);
    ElementLink(last, videologo);
    last = videologo;
  }

Your solution with pipeline = GST_ELEMENT (gst_parse_launch (pipe_desc->str, &error)); , not flexible for big pipelines, for example i have cases where i need rebuild pipelines on fly. How can I do this with gst_parse_launch function? For end-users better to have finished element not bicycle with tee and pads.

Here pipeline which we have: encode , I want to insert your elements after raw image (prev video_convert_0) right? how can I do this if I use low-level gst_element_link/gst_bin_add api?

[carlos-aguero]

Hi @topilski,

Thank you very much for your recommendations, we highly appreciate your feedback, currently we don't consider using a bin to contain GstInference and other required elements because we use GstInference on different architectures depending on the architecture elements vary, currently we prefer to have more flexibility with pipelines and elements that are used with GstInference.

If you are writting a GStreamer application that creates a pipeline encode it is possible to modify that pipeline even if it was created with gst_parse_launch, one option is to obtain references to pipeline elements and perform state changes, link, unlinks, etc. Similar to your provided code, but using a gst_bin_get_by_name ant others. On our case we also use GSTD-1.x and GstInterpipes for creating mediaservers.

If required please don't hesitate to contact us by using our Contact Us - RidgeRun and also please visit RidgeRun Site

Regards.

[topilski]

I know about gst_bin_get_by_name functionality we use it in our project, all GStreamer elements and even in ugly plugins can be used in both ways (gst_parse_launch and gst_element_link/gst_bin_add), thank you for your answers.