ros-perception / vision_opencv

Apache License 2.0
536 stars 599 forks source link

feat: extend type adaptation to support cv::cuda::GpuMat #519

Open AiVerisimilitude opened 5 months ago

AiVerisimilitude commented 5 months ago

Extended ROSCvMatContainer to support type adaptation for cv::cuda::GpuMat.

Internally I changed the frame type to a std::variant so the user can store either cv::Mat or cv::cuda::GpuMat. The user is responsible for checking the stored type first via holds_cv_type.

The use case here is for a pipeline that can use hardware acceleration across all nodes. Being able to pass cv::cuda::GpuMat is thus a requirement. The reasoning for having it type adapted together with cv::Mat is for interoperability.

The constructors from cv types were also condensed to a "pass by value and move" in order to have only 2 constructors (cv::Mat and cv::cuda::Mat) instead of 4(cv::Mat const &; cv::Mat &&; cv::cuda::GpuMat const &; cv::cuda::GpuMat &&)

[!WARNING] Usage of cv::cuda::GpuMat requires building of OpenCV with the CUDA module. I am not aware of this package being built with such module in mind but I am fine with it being a requirement that you pull & build the package as part of your workspace. Let me know your thoughts on this.

Example usage

creating a subscriber:

publisher = this->create_publisher<ROSCvMatContainer::AdaptedType>(topicName, 10);

getting the info:

void MyNode::image_topic_callback(std::unique_ptr<ROSCvMatContainer> image) {
  cv::cuda::GpuMat rbgMat;

  if (image->holds_cv_type<cv::Mat>()) {
    auto cvImage = image->cv_mat();
    rbgMat.upload(cvImage);
  } else if (image->holds_cv_type<cv::cuda::GpuMat>()) {
    rbgMat = image->cv_gpu_mat();
  } else {
    throw std::runtime_error("Unsupported image type");
  }
  ...

publishing:

  cv::cuda::GpuMat gpuImage;
  ...
  std_msgs::msg::Header header;
  header.stamp = now();

  auto result = std::make_unique<ROSCvMatContainer>(std::move(gpuImage), header);
  publisher->publish(std::move(result));