YanjieZe / 3D-Diffusion-Policy

[RSS 2024] 3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations
https://3d-diffusion-policy.github.io
MIT License
404 stars 37 forks source link

point cloud #46

Closed kingchou007 closed 3 months ago

kingchou007 commented 3 months ago

Hi,

Thanks for the great work.

I am currently working on processing real robot data and have a question about point preprocessing. In the paper, it is mentioned that depth images are obtained with a size of 84 × 84 from a single camera, and then downsampling use the point 1024. I would like to know the original size of the images before running the preprocess_image function in the script.

I want to ensure that I am following the correct workflow. I have RGB images with dimensions (H, W, 3) and corresponding depth images with dimensions (H, W). My current approach involves resizing the depth images (run the process_image function in the scripts) and then creating the point cloud from these resized depth images.

However, after resizing, the depth images appear incorrect. If I use the raw depth (1280, 720), it looks fine. Could you clarify the proper method for processing the depth images and generating point clouds?

Thanks

YanjieZe commented 3 months ago

Hi, thank you for your interest.

I think you could directly use your raw depth (so that the point cloud is right) and do the downsampling. Remember to do some crop before downsampling, to make the downsampled points more useful

Kimagure7 commented 2 months ago

Hi, 你好,

Thanks for the great work.感谢您做的伟大工作。

I am currently working on processing real robot data and have a question about point preprocessing. In the paper, it is mentioned that depth images are obtained with a size of 84 × 84 from a single camera, and then downsampling use the point 1024. I would like to know the original size of the images before running the preprocess_image function in the script.我目前正在处理真实的机器人数据,并对点预处理有疑问。论文中提到,从单个相机获得尺寸为84×84的深度图像,然后使用点1024进行下采样。我想在运行中的preprocess_image函数之前知道图像的原始尺寸脚本。

I want to ensure that I am following the correct workflow. I have RGB images with dimensions (H, W, 3) and corresponding depth images with dimensions (H, W). My current approach involves resizing the depth images (run the process_image function in the scripts) and then creating the point cloud from these resized depth images.我想确保我遵循正确的工作流程。我有尺寸为 (H, W, 3) 的 RGB 图像和尺寸为 (H, W) 的相应深度图像。我当前的方法包括调整深度图像的大小(运行脚本中的 process_image 函数),然后从这些调整大小的深度图像创建点云。

However, after resizing, the depth images appear incorrect. If I use the raw depth (1280, 720), it looks fine. Could you clarify the proper method for processing the depth images and generating point clouds?但是,调整大小后,深度图像显示不正确。如果我使用原始深度(1280, 720),它看起来不错。您能否阐明处理深度图像和生成点云的正确方法?

Thanks 谢谢

How do you generate PointCloud data in real world data? Did you write the script yourself?

kingchou007 commented 2 months ago

Hi, 你好, Thanks for the great work.感谢您做的伟大工作。 I am currently working on processing real robot data and have a question about point preprocessing. In the paper, it is mentioned that depth images are obtained with a size of 84 × 84 from a single camera, and then downsampling use the point 1024. I would like to know the original size of the images before running the preprocess_image function in the script.我目前正在处理真实的机器人数据,并对点预处理有疑问。论文中提到,从单个相机获得尺寸为84×84的深度图像,然后使用点1024进行下采样。我想在运行中的preprocess_image函数之前知道图像的原始尺寸脚本。 I want to ensure that I am following the correct workflow. I have RGB images with dimensions (H, W, 3) and corresponding depth images with dimensions (H, W). My current approach involves resizing the depth images (run the process_image function in the scripts) and then creating the point cloud from these resized depth images.我想确保我遵循正确的工作流程。我有尺寸为 (H, W, 3) 的 RGB 图像和尺寸为 (H, W) 的相应深度图像。我当前的方法包括调整深度图像的大小(运行脚本中的 process_image 函数),然后从这些调整大小的深度图像创建点云。 However, after resizing, the depth images appear incorrect. If I use the raw depth (1280, 720), it looks fine. Could you clarify the proper method for processing the depth images and generating point clouds?但是,调整大小后,深度图像显示不正确。如果我使用原始深度(1280, 720),它看起来不错。您能否阐明处理深度图像和生成点云的正确方法? Thanks 谢谢

How do you generate PointCloud data in real world data? Did you write the script yourself?

Hi,

You can take a look this example.

Kimagure7 commented 2 months ago

Hi, 你好, Thanks for the great work.感谢您做的伟大工作。 I am currently working on processing real robot data and have a question about point preprocessing. In the paper, it is mentioned that depth images are obtained with a size of 84 × 84 from a single camera, and then downsampling use the point 1024. I would like to know the original size of the images before running the preprocess_image function in the script.我目前正在处理真实的机器人数据,并对点预处理有疑问。论文中提到,从单个相机获得尺寸为84×84的深度图像,然后使用点1024进行下采样。我想在运行中的preprocess_image函数之前知道图像的原始尺寸脚本。 I want to ensure that I am following the correct workflow. I have RGB images with dimensions (H, W, 3) and corresponding depth images with dimensions (H, W). My current approach involves resizing the depth images (run the process_image function in the scripts) and then creating the point cloud from these resized depth images.我想确保我遵循正确的工作流程。我有尺寸为 (H, W, 3) 的 RGB 图像和尺寸为 (H, W) 的相应深度图像。我当前的方法包括调整深度图像的大小(运行脚本中的 process_image 函数),然后从这些调整大小的深度图像创建点云。 However, after resizing, the depth images appear incorrect. If I use the raw depth (1280, 720), it looks fine. Could you clarify the proper method for processing the depth images and generating point clouds?但是,调整大小后,深度图像显示不正确。如果我使用原始深度(1280, 720),它看起来不错。您能否阐明处理深度图像和生成点云的正确方法? Thanks 谢谢

How do you generate PointCloud data in real world data? Did you write the script yourself?

Hi,

You can take a look this example.

I haven't gotten to that point yet, I'm still struggling with generating point clouds from RGB-D pics. Can you share the code about how you handle this? 2477036173@qq.com. BTW, have you succeed in deploying dp3 in a real robot?