help in video streaming using dragon

[vishal-ahirwar]

Hey So, I'm building a video calling app and the prototype i made in python but for the production level i want use c++ for the performance and I have two alternatives framework to use dragon and the other one oat++; anyone please guide me how i can do video streaming with additional filter on server side using opencv ??

[hwc0919]

Not familiar with video processing. But sounds like you need tcp/udp to do the job? Drogon provides tcp client/server, but no udp support for now.

[vaizq]

I think you could use web sockets for streaming and drogon seems to support websockets.

[GuardHer]

This can be achieved using HttpResponse::newAsyncStreamResponse. The specific code is:

void video::async_video_stream(const HttpRequestPtr &req, std::function<void(const HttpResponsePtr &)> &&callback) const
{
    cap_.open(path); // can be modified to your camera
    if (!cap_.isOpened())
    {
        auto resp_err = HttpResponse::newHttpResponse();
        resp_err->setStatusCode(k500InternalServerError);
        resp_err->setBody("Failed to open video file");
        callback(resp_err);
        return;
    }
    auto sendFrame = [this](ResponseStreamPtr res) -> void
    {
        while (cap_.isOpened())
        {
            cv::Mat frame;
            cap_ >> frame; // Capture a frame
            if (!frame.empty())
            {
                std::vector<uchar> img_buffer;
                cv::imencode(".jpg", frame, img_buffer);
                std::string ima_data(img_buffer.begin(), img_buffer.end());
                auto img_size = img_buffer.size();
                std::ostringstream ss;
                ss << "--frame\r\nContent-Type: image/jpeg\r\n\r\n";
                ss << ima_data << "\r\n";
                res->send(ss.str());

                std::this_thread::sleep_for(std::chrono::milliseconds(10)); // 10ms
            }
            else
            {
                break;
            }
        }
        res->close();
        cap_.release();
    };

    auto resp = HttpResponse::newAsyncStreamResponse(sendFrame);
    resp->addHeader("Content-Type", "multipart/x-mixed-replace;boundary=frame");
    callback(resp);
}

[vishal-ahirwar]

This can be achieved using HttpResponse::newAsyncStreamResponse. The specific code is:

void video::async_video_stream(const HttpRequestPtr &req, std::function<void(const HttpResponsePtr &)> &&callback) const
{
  cap_.open(path); // can be modified to your camera
  if (!cap_.isOpened())
  {
      auto resp_err = HttpResponse::newHttpResponse();
      resp_err->setStatusCode(k500InternalServerError);
      resp_err->setBody("Failed to open video file");
      callback(resp_err);
      return;
  }
  auto sendFrame = [this](ResponseStreamPtr res) -> void
  {
      while (cap_.isOpened())
      {
          cv::Mat frame;
          cap_ >> frame; // Capture a frame
          if (!frame.empty())
          {
              std::vector<uchar> img_buffer;
              cv::imencode(".jpg", frame, img_buffer);
              std::string ima_data(img_buffer.begin(), img_buffer.end());
              auto img_size = img_buffer.size();
              std::ostringstream ss;
              ss << "--frame\r\nContent-Type: image/jpeg\r\n\r\n";
              ss << ima_data << "\r\n";
              res->send(ss.str());

              std::this_thread::sleep_for(std::chrono::milliseconds(10)); // 10ms
          }
          else
          {
              break;
          }
      }
      res->close();
      cap_.release();
  };

  auto resp = HttpResponse::newAsyncStreamResponse(sendFrame);
  resp->addHeader("Content-Type", "multipart/x-mixed-replace;boundary=frame");
  callback(resp);
}

thank you I will try this