search

mic-rud / RABBIT-Transcoding

RABBIT: Live Transcoding of V-PCC Point Cloud Streams
Other
1 stars 2 forks source link

Unexpected Translation of Decoded Point Clouds #2

Closed WaterHyacinthInNANHU closed 9 months ago

WaterHyacinthInNANHU commented 9 months ago

Hi! I find a strange problem: the point cloud reconstructed by the decoder is somehow shifted by a distance compared to the original point cloud.

Reproduce:

  1. My environment:

    • Linux 5.15.0-91-generic #101~20.04.1-Ubuntu
    • gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
    • cmake version 3.26.1

  1. Download testing data at here. I used the Thaidancer sequence.

    ├── bin
├── cfg
├── data
├── convert_ply.py
└── transcode_test.sh
  2. # encode-transcode-decode-metric
chmod +x transcode_test.sh & ./transcode_test.sh
# optinal, convert the binary ply to text
python convert_ply.py

Results

PccAppMetrics outputs:

Metrics results 
WARNING: 6185 points with same coordinates found
Imported intrinsic resoluiton: 1023
Peak distance for PSNR: 1023
Point cloud sizes for org version, dec version, and the scaling ratio: 3197804, 2849499, 0.89108
1. Use infile1 (A) as reference, loop over A, use normals on B. (A->B).
   mse1      (p2point): 17213.6
   mse1,PSNR (p2point): 22.61
   c[0],    1         : 0.00925975
   c[1],    1         : 0.00182647
   c[2],    1         : 0.00221639
   c[0],PSNR1         : 20.334
   c[1],PSNR1         : 27.3839
   c[2],PSNR1         : 26.5435
2. Use infile2 (B) as reference, loop over B, use normals on A. (B->A).
   mse2      (p2point): 1.65798e+06
   mse2,PSNR (p2point): 2.77293
   c[0],    2         : 0.00881047
   c[1],    2         : 0.00548923
   c[2],    2         : 0.00506387
   c[0],PSNR2         : 20.55
   c[1],PSNR2         : 22.6049
   c[2],PSNR2         : 22.9552
4. Final (symmetric).
   mseF      (p2point): 1.65798e+06
   mseF,PSNR (p2point): 2.77293
   mseF      (p2plane): 0
   mseF,PSNR (p2plane): 0
   c[0],    F         : 0.00925975
   c[1],    F         : 0.00548923
   c[2],    F         : 0.00506387
   c[0],PSNRF         : 20.334
   c[1],PSNRF         : 22.6049
   c[2],PSNRF         : 22.9552
WARNING: 6034 points with same coordinates found
Imported intrinsic resoluiton: 1023
Peak distance for PSNR: 1023
Point cloud sizes for org version, dec version, and the scaling ratio: 3205574, 2868162, 0.894742
1. Use infile1 (A) as reference, loop over A, use normals on B. (A->B).
   mse1      (p2point): 17792.4
   mse1,PSNR (p2point): 22.4664
   c[0],    1         : 0.00899534
   c[1],    1         : 0.00173004
   c[2],    1         : 0.00208794
   c[0],PSNR1         : 20.4598
   c[1],PSNR1         : 27.6194
   c[2],PSNR1         : 26.8028
2. Use infile2 (B) as reference, loop over B, use normals on A. (B->A).
   mse2      (p2point): 1.66475e+06
   mse2,PSNR (p2point): 2.75525
   c[0],    2         : 0.00775946
   c[1],    2         : 0.00542392
   c[2],    2         : 0.0050103
   c[0],PSNR2         : 21.1017
   c[1],PSNR2         : 22.6569
   c[2],PSNR2         : 23.0014
3. Final (symmetric).
   mseF      (p2point): 1.66475e+06
   mseF,PSNR (p2point): 2.75525
   mseF      (p2plane): 0
   mseF,PSNR (p2plane): 0
   c[0],    F         : 0.00899534
   c[1],    F         : 0.00542392
   c[2],    F         : 0.0050103
   c[0],PSNRF         : 20.4598
   c[1],PSNRF         : 22.6569
   c[2],PSNRF         : 23.0014

Point cloud reconstructed by PccAppDecoder:

ply
format ascii 1.0
element vertex 2855684
property float x
property float y
property float z
property uchar red
property uchar green
property uchar blue
element face 0
property list uchar int vertex_index
end_header
1844 166 439 43 27 11
1845 166 438 42 26 9
1844 167 439 44 27 10
1845 167 438 44 26 9
1842 168 442 44 29 10
1843 168 441 45 29 10
1844 168 440 45 27 10
1845 168 439 45 26 9
1842 169 442 45 31 9
1843 169 441 46 30 10
1844 169 440 47 28 10
1845 169 439 48 28 10
1838 170 447 46 30 10
1839 170 446 45 30 10
1840 170 445 45 32 11
1841 170 444 45 32 10
1842 170 442 45 32 9
1843 170 441 47 31 10

Ground truth point cloud reconstructed by PccAppEncoder (print it instead of the raw point cloud for its better formatting):

ply
format ascii 1.0
element vertex 3239876
property float x
property float y
property float z
property uchar red
property uchar green
property uchar blue
element face 0
property list uint8 int32 vertex_index
end_header
1844 166 2486 45 26 9
1845 166 2486 45 26 9
1844 167 2487 46 27 9
1845 167 2486 47 27 9
1842 168 2490 48 28 9
1843 168 2488 48 28 9
1844 168 2487 47 27 8
1845 168 2487 47 27 8
1842 169 2490 50 29 9
1843 169 2489 50 29 9
1844 169 2488 49 28 8
1845 169 2487 49 28 8
1838 170 2495 51 29 8
1839 170 2494 51 29 8
1840 170 2493 51 29 8
1841 170 2492 52 30 9
1842 170 2491 52 30 9
1843 170 2490 52 30 9

It can be seen that the whole point cloud is "shifted" by a distance, leading to a low PSNR. (only ~6k points with same coordinates)

Is this behavior expected, or did I miss anything? Please help~

mic-rud commented 9 months ago

Hi, this is indeed strange behaviour. Looks like the point cloud is shifted in z-direction.

Could you share the code for _testtranscode.sh and _convertply.py?

Also, does this happen only for the full Encode -> Transcode -> Decode pipeline or do you observe the same behaviour when running only Encode -> Decode?

Edit: I found the files i asked for in your upload, configurations look good to me.

WaterHyacinthInNANHU commented 9 months ago

Thank you for the reply! I think I found the reason. The translation is always an exponent of 2, so it must be an integer overflowing error. I fixed it by setting geometry3dCoordinatesBitdepth from 11 to 12, which matches the 12-bit depth of the raw Thaidancer point cloud data. It is a problem with the TMC's configuration file.