What is 'Outside9.npy'?

[GitChanyoung]

Hello. I'm looking to use my custom dataset with rcvpose. What values are present in outside9.npy, which is needed for training?

[aaronWool]

Hi, It is the dispersed keypoint in the paper. I have updated the rmap_dataset so that those keypoints will be generated if 'Outside9.npy' does not exist. Thanks for pointing this out!

[GitChanyoung]

I sincerely appreciate your response. I have two additional questions. I'm attempting to download the LINEMOD dataset for training. However, I encounter the "ValueError: loss is nan while training" error

and upon inspecting the data_loader.py file, it appears that after the line: lbl = np.where(lbl > self.max_radii_dm, 0, lbl) all the values seem to become 0.

The above image is before running the code, and the one below is after running the code.

Furthermore, when conducting the test, there are a total of 9 keypoints, unlike the 3 keypoints used. I'm curious if it's acceptable to use something other than indexes 0, 1, and 2 or if I should stick with just 0, 1, and 2.

[aaronWool]

Hi,

Thanks for pointing out the bug, it should be fixed now.

The inference time only uses three keypoints for the results reported in the paper. All keypoints are saved for experiments purposes. The keypoints being used are index 1, 2, and 3. The index 0 is the center of the object. It would be interesting to see what would happen when using one center keypoint, keep me posted!

[GitChanyoung]

Hello, I'm amazed at how quickly you made the corrections. Thanks to that, I was able to successfully complete the training. However, I have some parts that I don't understand during the testing process, so I have a question.

In the process of applying ICP (gt vs est after icp), I used the GT value for AccumulatorSpace.py

criteria = o3d.pipelines.registration.ICPConvergenceCriteria(max_iteration=2000)
reg_p2p = o3d.pipelines.registration.registration_icp(
                    sceneGT, sceneEst, threshold, trans_init,
                    o3d.pipelines.registration.TransformationEstimationPointToPoint(),
                    criteria)
sceneGT.transform(reg_p2p.transformation)

Shouldn't the estimation process be evaluated without the GT value's involvement in assessing performance? If I'm misunderstanding something, I'd appreciate it if you could explain it to me.

[aaronWool]

Hi,

Nice catch! I did not even notice that and I was confused by myself. If you could pull the latest commit, you would see the mis-usage of GT info in ICP is removed. Thanks for your valuable input!

[GitChanyoung]

Hello, I thought I had successfully completed the training with your quick feedback and valuable assistance. However, it seems that's not the case.

When I actually ran the model, the CPU was at 100% usage, and the inference process took an incredibly long time. After some investigation, I found that the operations in AccumulatorSpace.py -> def fast_for are taking a long time.

Before using the custom dataset, I tried using the LINEMOD dataset.

The Outside9.npy I mentioned in my initial question is not the one you provided; I created a new one. [[ 3.19084318e-06 2.76674088e-07 -1.41040132e-06] [-3.46675391e-03 9.68441917e-02 1.17202536e-01] [ 4.54035882e-02 1.04469195e-01 -6.44897717e-02] [ 5.84116872e-02 -4.90366167e-02 1.27724054e-01] [-1.31444511e-01 3.99046689e-02 8.03904056e-02] [-2.06957281e-02 -9.83511358e-02 -9.07803838e-02] [-6.95660702e-02 -1.05976140e-01 9.09119236e-02] [-8.25741693e-02 4.75296726e-02 -1.01301902e-01] [ 1.07282029e-01 -4.14116130e-02 -5.39682537e-02]]

is the one you provided, and

[[ 8.99959987 -91.46225263 -184.72030625] [ 57.86295125 -99.14615832 -3.0296815 ] [ 71.38218809 54.13938243 -195.33963842] [-118.60853041 -34.10902856 -147.97621072] [ -7.3625908 103.80870081 23.09508186] [ -56.22594218 111.4926065 -158.59554289] [ -69.74517903 -41.79293425 33.71441403] [ 120.24553947 46.45547674 -13.64901367]]

is the one I created later after my initial question. It seems that the two Outside9.npy files I created are very different, even though they are both for the same "ape" class.

The problem is that the units are significantly different between the two, and although I thought it wouldn't matter, it seems there's an issue. Could you please explain how the Outside9.npy you provided was generated?

[aaronWool]

Hi,

Yes, the nested for loop is the accumulator space and it can be stuck if the keypoints (Outside9.npy) are in the wrong unit.

You can use any keypoints you wish, Outside9.npy is defined by the 2* oriented BBox of the object using Open3D.

---

From: GitChanyoung @.> Sent: 18 October 2023 04:31 To: aaronWool/rcvpose @.> Cc: YANGZHENG WU @.>; Comment @.> Subject: Re: [aaronWool/rcvpose] What is 'Outside9.npy'? (Issue #16)

Hello, I thought I had successfully completed the training with your quick feedback and valuable assistance. However, it seems that's not the case.

When I actually ran the model, the CPU was at 100% usage, and the inference process took an incredibly long time. After some investigation, I found that the operations in AccumulatorSpace.py -> def fast_for are taking a long time.

Before using the custom dataset, I tried using the LINEMOD dataset.

The Outside9.npy I mentioned in my initial question is not the one you provided; I created a new one. [[ 3.19084318e-06 2.76674088e-07 -1.41040132e-06] [-3.46675391e-03 9.68441917e-02 1.17202536e-01] [ 4.54035882e-02 1.04469195e-01 -6.44897717e-02] [ 5.84116872e-02 -4.90366167e-02 1.27724054e-01] [-1.31444511e-01 3.99046689e-02 8.03904056e-02] [-2.06957281e-02 -9.83511358e-02 -9.07803838e-02] [-6.95660702e-02 -1.05976140e-01 9.09119236e-02] [-8.25741693e-02 4.75296726e-02 -1.01301902e-01] [ 1.07282029e-01 -4.14116130e-02 -5.39682537e-02]]

is the one you provided, and

[[ 8.99959987 -91.46225263 -184.72030625] [ 57.86295125 -99.14615832 -3.0296815 ] [ 71.38218809 54.13938243 -195.33963842] [-118.60853041 -34.10902856 -147.97621072] [ -7.3625908 103.80870081 23.09508186] [ -56.22594218 111.4926065 -158.59554289] [ -69.74517903 -41.79293425 33.71441403] [ 120.24553947 46.45547674 -13.64901367]]

is the one I created later after my initial question. It seems that the two Outside9.npy files I created are very different, even though they are both for the same "ape" class.

The problem is that the units are significantly different between the two, and although I thought it wouldn't matter, it seems there's an issue. Could you please explain how the Outside9.npy you provided was generated?

— Reply to this email directly, view it on GitHubhttps://github.com/aaronWool/rcvpose/issues/16#issuecomment-1767953513, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AMRZJMUGEUZNHDLSTMQJ7GTX76HXFAVCNFSM6AAAAAA5VTATJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRXHE2TGNJRGM. You are receiving this because you commented.Message ID: @.***>

[KarelZhang]

Hi, @GitChanyoung @aaronWool When I run official checkpoint of the cat object of LINEMOD, I got ADD(S) zero, I noticed that the min distance is round 2 cm, which is quite large. Is there any suggestion? The dataset I used is downloaded from this And I modified some bugs in test code, which is the depth is from LINEMOD instead of LINEMOD_ORIG AccumulatorSpace.py def estimate_6d_pose_lm():

Thanks a lot for helping me !!!

[aaronWool]

Hi Jiaoao,

The depth value has a different scale factor. You will need to apply the factor to make it work, if you changed the default depth path.

Aaron Wu

On Tue, Jan 23, 2024, 8:57 a.m. Jiaao Zhang @.***> wrote:

Hi, @GitChanyoung https://github.com/GitChanyoung @aaronWool https://github.com/aaronWool When I run official checkpoint of the cat object of LINEMOD, I got ADD(S) zero, I noticed that the min distance is round 2 cm, which is quite large. Is there any suggestion? The dataset I used is downloaded from this https://queensuca-my.sharepoint.com/:u:/g/personal/16yw113_queensu_ca/EcDsKd-cPIhDrfgfmeHnjCkBnd84Vxe8oklS8VEm5c3cug?e=E8bXcR And I modified some bugs in test code, which is the depth is from LINEMOD instead of LINEMOD_ORIG default.png (view on web) https://github.com/aaronWool/rcvpose/assets/26562112/2124a65d-f569-4d67-81ea-76e4891d6959 Thanks a lot for helping me !!!

— Reply to this email directly, view it on GitHub https://github.com/aaronWool/rcvpose/issues/16#issuecomment-1906114065, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMRZJMS46DL47WNKFPLLSX3YP66U3AVCNFSM6AAAAAA5VTATJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBWGEYTIMBWGU . You are receiving this because you were mentioned.Message ID: @.***>

[KarelZhang]

Hi Aaron, @aaronWool

Thanks for your reply. The code I changed is just read depth from LINEMOD_ORIG to LINEMOD. Here is the code: And I did not change the read_depth function: Is there anything wrong?

Jiaao