Saving PCD files consumes memory after writing the file

sjdrc commented 7 years ago

:warning: This is a issue tracker, please use our mailing list for questions: www.pcl-users.org. :warning:

Your Environment

Operating System and version: Jetson TX1 Ubuntu 16.04
Compiler: 5.4.0
PCL Version: 1.8 (from git)

Current Behavior

Using pcl::io::savePCDFile() consumes as much RAM as the filesize. As I am running this in a loop, this quickly consumes all memory until the program crashes. The memory is freed when I delete the files from disk while the program is running, and some of it is freed when I terminate the program. This occurs with all methods of saving PCD files.

Expected Behavior

Saving PCD files to disk should not consume all the RAM even after the file has been written

Code to Reproduce

    int savePersonCluster(PointCloud::ConstPtr cloud, pcl::people::PersonCluster<PointT> cluster, std::string uuid) {
        PointCloud::Ptr tmp(new PointCloud);
        pcl::CropBox<PointT> crop;
        crop.setInputCloud(cloud);
        crop.setMin(Eigen::Vector4f(cluster.getMin().x(), cluster.getMin().y(), cluster.getMin().z(), 0));
        crop.setMax(Eigen::Vector4f(cluster.getMax().x(), cluster.getMax().y(), cluster.getMax().z(), 0));
        crop.filter(*tmp);
        tmp->header.frame_id = cloud->header.frame_id;
        std::stringstream ss;
        ss << cluster_save_path << "/" << uuid << std::string(".pcd");
        if (save_person_clusters) pcl::io::savePCDFileBinaryCompressed(ss.str(), *tmp);
        return tmp->points.size();
    }

Setting save_person_clusters to false does not consume any memory.

Context

I am pulling pointclouds from a Kinect v2 using iai_kinect2 using ROS.

taketwo commented 7 years ago

I do see potential memory leaks in writeBinaryCompressed(), because malloced memory is freed only when all I/O operations are successful. There are a bunch execution paths that leave the function without freeing the memory. Do you think you might be actually getting into one of these cases?

sjdrc commented 7 years ago

Yeah that definitely seems like what is happening. I'm writing to an ext4 mounted filesystem with 755 permissions owned by the same user that runs the program. We did have a similar system running about a month ago that managed to save a lot more data than what our current system can, but unfortunately that board died so I can't check and see why there was no memory leak there.

We're running the program on a Jetson TX1 embedded system so it's a little be hard for me to debug this, recompiling PCL with debugging symbols takes a few hours and I don't have an IDE running on there to easily troubleshoot this. Given that the files actually get written and contain valid data, at what point do you think it is erroring out? Any other suggestions on how to check this out?

Thanks for the fast reply!

EDIT: Also this leak does occur with savePCDFileASCII()

taketwo commented 7 years ago

Weird, there are no memory allocations in ASCII version. No idea.

kaisark commented 6 years ago

@sjdrc - I see you were able to build pcl 1.8 from source on the Jetson TX1. Can you please post which instructions you followed?

TX1 comes with libpcl1.7 in ubuntu repositories, but I would like to work with the latest version. I am trying to get a similar setup working on the TX1 (Kinect v2 using iai_kinect2 using ROS). Libfreenect2 and OpenNI are both working...

Thanks.

brunoeducsantos commented 6 years ago

Hey, Has anyone sorted out the issue? Cheers, Bruno

PointCloudLibrary / pcl