About the application of simulation data model in real scenes

[cy-coder]

Hello, seniors, I am currently using PPR-Net to conduct 6D pose estimation experiments for industrial parts. I have the following questions. I want to ask you

First, whether it is necessary to restore the simulated data to be consistent with the real scene when making simulation data, such as camera internal parameters.

Second, I tried a variety of data sets under different simulated camera parameters, such as modifying the internal parameters of the camera and the size of the workpiece. The simulation prediction results are good or bad. Compared with the high-precision results of the public data set, I would like to ask how I can design the simulation data set to get better results.

Third, I set the simulation data set to randomly drop one by one workpiece, and record the scene pictures after each fall. This method is not consistent with the public data set. I found that every picture in the public data set seems to be different. The same scene. Is there something wrong with my approach?

I look forward to your answer to my question

[xumingkun9806]

@cy-coderb Can you explain how the simulation dataset was generated?for example,blender or v- rep?Thank you very much!!! Then,I think ppr-net training is the point cloud,This has nothing to do with camera internal parameters?

[xumingkun9806]

@cy-coder type_bunny = ObjectType(type_name='bunny', class_idx=0, symmetry_type='finite', lambda_p=[[0.0263663, 0.0, 0.0], [0.0, 0.0338224, 0.0], [-0.0, 0.0, 0.0484393]], G=[ [[1,0,0], [0,1,0], [0,0,1]] ]) and if you know this lamdba_p is waht?

[cy-coder]

Can I add a WeChat or QQ to chat in detail?  I will explain my detailed simulation situation to you. The type_bunny is not clear yet. The original one is used when running the data set. My QQ is 734789965 and WeChat is cuiy951111 thank you very much

------------------ 原始邮件 ------------------ 发件人: "lvwj19/PPR-Net-plus" @.>; 发送时间: 2021年11月19日(星期五) 下午3:42 @.>; @.**@.>; 主题: Re: [lvwj19/PPR-Net-plus] About the application of simulation data model in real scenes (Issue #1)

@cy-coder type_bunny = ObjectType(type_name='bunny', class_idx=0, symmetry_type='finite', lambda_p=[[0.0263663, 0.0, 0.0], [0.0, 0.0338224, 0.0], [-0.0, 0.0, 0.0484393]], G=[ [[1,0,0], [0,1,0], [0,0,1]] ]) and if you know this lamdba_p is waht?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[cy-coder]

I use the simulated data generated by v-rep. I also recently watched PPR-Net++. The article mentions the method of simulating the real one, one of which is to closely match the simulated data with the real data. What I input is point cloud information, but different simulated cameras will generate simulation data that is different from the real scene. When I use the simulation data model to predict the real point cloud, the effect is not ideal, so I want to use the simulation data to approximate the real point cloud Scenes

------------------ 原始邮件 ------------------ 发件人: "lvwj19/PPR-Net-plus" @.>; 发送时间: 2021年11月19日(星期五) 下午3:38 @.>; @.**@.>; 主题: Re: [lvwj19/PPR-Net-plus] About the application of simulation data model in real scenes (Issue #1)

@cy-coderb Can you explain how the simulation dataset was generated?for example,blender or v- rep?Thank you very much!!! Then,I think ppr-net training is the point cloud,This has nothing to do with camera internal parameters?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[xumingkun9806]

@cy-coder ### 加了你的微信,通过下

[lvwj19]

Hello, seniors, I am currently using PPR-Net to conduct 6D pose estimation experiments for industrial parts. I have the following questions. I want to ask you

First, whether it is necessary to restore the simulated data to be consistent with the real scene when making simulation data, such as camera internal parameters.

Second, I tried a variety of data sets under different simulated camera parameters, such as modifying the internal parameters of the camera and the size of the workpiece. The simulation prediction results are good or bad. Compared with the high-precision results of the public data set, I would like to ask how I can design the simulation data set to get better results.

Third, I set the simulation data set to randomly drop one by one workpiece, and record the scene pictures after each fall. This method is not consistent with the public data set. I found that every picture in the public data set seems to be different. The same scene. Is there something wrong with my approach?

I look forward to your answer to my question

Thank you for your attention. Firstly, we keep consistent for camera internal parameters between the simulated and real data. Secondly, we use domain randomization when training with simulated data. Thirdly, we drop all objects together randomly for each scene, like IPA dataset (IROS2019 Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-Picking), which means that each scene is different.

[lvwj19]

@cy-coder type_bunny = ObjectType(type_name='bunny', class_idx=0, symmetry_type='finite', lambda_p=[[0.0263663, 0.0, 0.0], [0.0, 0.0338224, 0.0], [-0.0, 0.0, 0.0484393]], G=[ [[1,0,0], [0,1,0], [0,0,1]] ]) and if you know this lamdba_p is waht?

As for the lambda, we calculate the value of lambda according to the paper (Defining the Pose of any 3D Rigid Object and an Associated Distance). We have provided the code for calculation in “models/calc_lambda.py”

[cy-coder]

Thank you very much for replying to me so quickly. Regarding the construction of the simulation data set, I am still new to contact, and I have encountered many problems on my own. Because I don’t know the specific construction method of the public data set, such as the segmentation map, I used v-rep to use some alternative methods to create a similar data set. I have used my own simulation artifacts for many tests, and the accuracy of the results is not ideal. The best prediction loss can only reach 11, and the worst can reach 30. I want to hear how to adjust the simulation data set to reduce the loss. Your understanding. If possible, can I add a WeChat account. My WeChat is cuiy951111. I would like to explain to you my simulation data in detail. I am very grateful.

------------------ 原始邮件 ------------------ 发件人: "lvwj19/PPR-Net-plus" @.>; 发送时间: 2021年11月19日(星期五) 下午4:38 @.>; @.**@.>; 主题: Re: [lvwj19/PPR-Net-plus] About the application of simulation data model in real scenes (Issue #1)

Hello, seniors, I am currently using PPR-Net to conduct 6D pose estimation experiments for industrial parts. I have the following questions. I want to ask you

First, whether it is necessary to restore the simulated data to be consistent with the real scene when making simulation data, such as camera internal parameters.

Second, I tried a variety of data sets under different simulated camera parameters, such as modifying the internal parameters of the camera and the size of the workpiece. The simulation prediction results are good or bad. Compared with the high-precision results of the public data set, I would like to ask how I can design the simulation data set to get better results.

Third, I set the simulation data set to randomly drop one by one workpiece, and record the scene pictures after each fall. This method is not consistent with the public data set. I found that every picture in the public data set seems to be different. The same scene. Is there something wrong with my approach?

I look forward to your answer to my question

Thank you for your attention. Firstly, we keep consistent for camera internal parameters between the simulated and real data. Secondly, we use domain randomization when training with simulated data. Thirdly, we drop all objects together randomly for each scene, like IPA dataset (IROS2019 Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-Picking), which means that each scene is different.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[lvwj19]

Thank you very much for replying to me so quickly. Regarding the construction of the simulation data set, I am still new to contact, and I have encountered many problems on my own. Because I don’t know the specific construction method of the public data set, such as the segmentation map, I used v-rep to use some alternative methods to create a similar data set. I have used my own simulation artifacts for many tests, and the accuracy of the results is not ideal. The best prediction loss can only reach 11, and the worst can reach 30. I want to hear how to adjust the simulation data set to reduce the loss. Your understanding. If possible, can I add a WeChat account. My WeChat is cuiy951111. I would like to explain to you my simulation data in detail. I am very grateful. … ------------------ 原始邮件 ------------------ 发件人: "lvwj19/PPR-Net-plus" @.>; 发送时间: 2021年11月19日(星期五) 下午4:38 @.>; @.**@.>; 主题: Re: [lvwj19/PPR-Net-plus] About the application of simulation data model in real scenes (Issue #1) Hello, seniors, I am currently using PPR-Net to conduct 6D pose estimation experiments for industrial parts. I have the following questions. I want to ask you First, whether it is necessary to restore the simulated data to be consistent with the real scene when making simulation data, such as camera internal parameters. Second, I tried a variety of data sets under different simulated camera parameters, such as modifying the internal parameters of the camera and the size of the workpiece. The simulation prediction results are good or bad. Compared with the high-precision results of the public data set, I would like to ask how I can design the simulation data set to get better results. Third, I set the simulation data set to randomly drop one by one workpiece, and record the scene pictures after each fall. This method is not consistent with the public data set. I found that every picture in the public data set seems to be different. The same scene. Is there something wrong with my approach? I look forward to your answer to my question Thank you for your attention. Firstly, we keep consistent for camera internal parameters between the simulated and real data. Secondly, we use domain randomization when training with simulated data. Thirdly, we drop all objects together randomly for each scene, like IPA dataset (IROS2019 Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-Picking), which means that each scene is different. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

We use Bullet and Blender to generate the simulated data. Sorry, we have no experience with v-rep. As for details of our methods, you can refer to our patents("数据处理方法及装置", "多种类工业零件堆叠场景的仿真数据集生成方法及装置", "物体位姿识别的方法、装置及计算机存储介质 ").

[cy-coder]

Thank you very much and look forward to the next exchange with you

------------------ 原始邮件 ------------------ 发件人: "lvwj19/PPR-Net-plus" @.>; 发送时间: 2021年11月19日(星期五) 下午5:22 @.>; @.**@.>; 主题: Re: [lvwj19/PPR-Net-plus] About the application of simulation data model in real scenes (Issue #1)

Thank you very much for replying to me so quickly. Regarding the construction of the simulation data set, I am still new to contact, and I have encountered many problems on my own. Because I don’t know the specific construction method of the public data set, such as the segmentation map, I used v-rep to use some alternative methods to create a similar data set. I have used my own simulation artifacts for many tests, and the accuracy of the results is not ideal. The best prediction loss can only reach 11, and the worst can reach 30. I want to hear how to adjust the simulation data set to reduce the loss. Your understanding. If possible, can I add a WeChat account. My WeChat is cuiy951111. I would like to explain to you my simulation data in detail. I am very grateful. … ------------------�0�2原始邮件�0�2------------------ 发件人: "lvwj19/PPR-Net-plus" @.>; 发送时间:�0�22021年11月19日(星期五) 下午4:38 @.>; @.@.>; 主题:�0�2Re: [lvwj19/PPR-Net-plus] About the application of simulation data model in real scenes (Issue #1) Hello, seniors, I am currently using PPR-Net to conduct 6D pose estimation experiments for industrial parts. I have the following questions. I want to ask you First, whether it is necessary to restore the simulated data to be consistent with the real scene when making simulation data, such as camera internal parameters. Second, I tried a variety of data sets under different simulated camera parameters, such as modifying the internal parameters of the camera and the size of the workpiece. The simulation prediction results are good or bad. Compared with the high-precision results of the public data set, I would like to ask how I can design the simulation data set to get better results. Third, I set the simulation data set to randomly drop one by one workpiece, and record the scene pictures after each fall. This method is not consistent with the public data set. I found that every picture in the public data set seems to be different. The same scene. Is there something wrong with my approach? I look forward to your answer to my question Thank you for your attention. Firstly, we keep consistent for camera internal parameters between the simulated and real data. Secondly, we use domain randomization when training with simulated data. Thirdly, we drop all objects together randomly for each scene, like IPA dataset (IROS2019 Large-scale 6D Object Pose Estimation Dataset for Industrial Bin-Picking), which means that each scene is different. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

We use Bullet and Blender to generate the simulated data. Sorry, we have no experience with v-rep. As for details of our methods, you can refer to our patents("数据处理方法及装置", "多种类工业零件堆叠场景的仿真数据集生成方法及装置", "物体位姿识别的方法、装置及计算机存储介质 ").

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[trungpham2606]

@cy-coder Can you show some of the results (good and bad) ?

[cy-coder]

Maybe we can add a contact method for more in-depth communication

------------------ 原始邮件 ------------------ 发件人: "lvwj19/PPR-Net-plus" @.>; 发送时间: 2021年11月21日(星期天) 晚上8:39 @.>; @.**@.>; 主题: Re: [lvwj19/PPR-Net-plus] About the application of simulation data model in real scenes (Issue #1)

@cy-coder Can you show some of the results (good and bad) ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[trungpham2606]

@cy-coder here is my gmail: trungpham2606@gmail.com and my telegram: @trungpham2606