search

BrownBiomechanics / Autoscoper

Autoscoper is a 2D-3D image registration software package.
https://autoscoper.readthedocs.io
Other
8 stars 5 forks source link

inconsistent NCC values - AutoscoperMainWindow.cpp Socket.cpp #82

Closed amymmorton closed 1 year ago

amymmorton commented 1 year ago

The ncc values using AutoscoperCOnnection.m getNCC_ThisFrame are not the same for a given tracked file as the AutoscoperMainWIndow::save_ncc_results

ncc_disagreemntExport

amymmorton commented 1 year ago

NCC values differ opencl / cuda too: ncc_ocl_cuda

jcfr commented 1 year ago

save_ncc_results (*.ncc file)

https://github.com/BrownBiomechanics/Autoscoper/blob/56683fd2b2f729f0020d9c02538ad5c3b50bdabb/autoscoper/src/ui/AutoscoperMainWindow.cpp#L2057-L2079

getNCC (through socket)

https://github.com/BrownBiomechanics/Autoscoper/blob/91e7bc9a76a524f9f373ca4324e2915e6e75999b/autoscoper/src/ui/AutoscoperMainWindow.cpp#L908-L911

https://github.com/BrownBiomechanics/Autoscoper/blob/91e7bc9a76a524f9f373ca4324e2915e6e75999b/autoscoper/src/net/Socket.cpp#L212-L227

jcfr commented 1 year ago

file vs socket

@amymmorton Since there is one NCC value computed for each "view" and in both case (file vs socket) only the first two values are expected to match, which value did you plot (Figure 7 above) ?

Values
*.ncc correlation_0, correlation_1, correlation_0 + correlation_1
socket correlation_0, correlation_1 [, ..., correlation_N]

Cuda vs OpenCL

Worth noting that if we do Implant and are using the OpenCL backend, no value is computed. See below. I am wondering if this could explain some of the differences.

Comparing the input poses used for the tracking in both case may also help understand better.

https://github.com/BrownBiomechanics/Autoscoper/blob/91e7bc9a76a524f9f373ca4324e2915e6e75999b/libautoscoper/src/Tracker.cpp#L636-L650

amymmorton commented 1 year ago

@jcfr These ncc values were exported from a 'ground truth' set of poses. No optimization was performed / this trial was not re-tracked.

'only the first two values are expected to match, which value did you plot (Figure 7 above) ?' All three values were plotted, but you are correct, we should only concentration the first two returned outputs. Please see below ncc_comparisonSocketMain a revised format of the ncc values output pairs of the same data from above.

amymmorton commented 1 year ago

flx_ext_mc3_ncc_socket.csv flx_ext_mc3_ncc_cuda.csv flx_ext_mc3_ncc_opencl.csv

NicerNewerCar commented 1 year ago

NCC Computations

@amymmorton Let me know if you have any questions!

Tracker

This is the function used to compute the NCC values for a given frame (both for the PSO and save_ncc functions). This renders the radiographs DRR, and mask into Buffers. It then takes those buffers and runs the ncc calculations through the gpu. https://github.com/BrownBiomechanics/Autoscoper/blob/91e7bc9a76a524f9f373ca4324e2915e6e75999b/libautoscoper/src/Tracker.cpp#L590-L651

NCC

Note: Whenever buffers f and g appear together they are the DRR and the Radiograph respectively. If just Buffer f appears then it is used on both the DRR and Radiograph individually.

OpenCL

First, the NCCSum kernel is run on both the radiograph and DRR buffers. This computes the means of each buffer. Once this is done the main NCC kernel is called. This computes the NCC values based on the Radiograph, DRR, and mask buffers and the means of the radiograph and the DRR. This is repeated for all views/bone combinations.

Sum Kernel

https://github.com/BrownBiomechanics/Autoscoper/blob/91e7bc9a76a524f9f373ca4324e2915e6e75999b/libautoscoper/src/gpu/opencl/kernel/NccSum.cl#L1-L26

Sum Driver

https://github.com/BrownBiomechanics/Autoscoper/blob/91e7bc9a76a524f9f373ca4324e2915e6e75999b/libautoscoper/src/gpu/opencl/Ncc.cpp#L108-L155

NCC Kernel

https://github.com/BrownBiomechanics/Autoscoper/blob/91e7bc9a76a524f9f373ca4324e2915e6e75999b/libautoscoper/src/gpu/opencl/kernel/Ncc.cl#L1-L28

NCC Driver

https://github.com/BrownBiomechanics/Autoscoper/blob/91e7bc9a76a524f9f373ca4324e2915e6e75999b/libautoscoper/src/gpu/opencl/Ncc.cpp#L188-L228

CUDA

Just like the OpenCL kernels the CUDA kernel computes the means of the radiograph and DRR buffers then calculates the NCC value.

Sum Kernel

https://github.com/BrownBiomechanics/Autoscoper/blob/91e7bc9a76a524f9f373ca4324e2915e6e75999b/libautoscoper/src/gpu/cuda/Ncc_kernels.cu#L177-L197

Sum Driver

https://github.com/BrownBiomechanics/Autoscoper/blob/91e7bc9a76a524f9f373ca4324e2915e6e75999b/libautoscoper/src/gpu/cuda/Ncc_kernels.cu#L157-L175

NCC Kernel

https://github.com/BrownBiomechanics/Autoscoper/blob/91e7bc9a76a524f9f373ca4324e2915e6e75999b/libautoscoper/src/gpu/cuda/Ncc_kernels.cu#L199-L220

NCC Driver

https://github.com/BrownBiomechanics/Autoscoper/blob/91e7bc9a76a524f9f373ca4324e2915e6e75999b/libautoscoper/src/gpu/cuda/Ncc_kernels.cu#L118-L137

NicerNewerCar commented 1 year ago

NCC reported from WN00105 MC3

NCC values are exported from the GUI, MATLAB, and Python.

Ground truth tracking data was loaded for MC3 and then exported with the GUI, MATLAB, and Python with no optimization being done. The differences were then taken for each camera in each frame. The absolute value of those differences was taken and then plotted.

CUDA

CUDA-GUI-vs-Python

CUDA-GUI-vs-Matlab

OpenCL

OpenCL-GUI-vs-Python

OpenCL-GUI-vs-Matlab

CUDA vs OpenCL - GUI

CUDA-vs-OpenCL-GUI

jcfr commented 1 year ago

CUDA Python and CUDA Matlab seems to match. This is good.

Next, could you instrument the code to display input and output values. By adding print statement to:

NicerNewerCar commented 1 year ago

Just looking at a single frame:

getNCC input:

volumeID: 1, pose: [114.096, 80.5325, 184.475, -168.469, 23.8443, 13.8261]

PyAutoscoper input:

volumeID: 1, pose: [114.0963134765625, 80.53250885009766, 184.47508239746094, -168.46871948242188, 23.844257354736328, 13.826091766357422]

trackFrame output:

1.00758 1.00321

PyAutoscoper output:

ncc: [1.0075845890678465, 1.0032108505256474]

This is consistent across every frame, they are the same values just the Autoscoper C++ code deals with smaller places than the Python code.

The corresponding value from the GUI: 0.474160,0.514982,0.989142

NicerNewerCar commented 1 year ago

The poses are identical between GUI and the socket too, here are the poses that provided the above results

GUI: frame: 399 pose: [114.096,80.5325,184.475,-168.469,23.8443,13.8261]

Socket C++: Frame: 399 Pose: [114.096, 80.5325, 184.475, -168.469, 23.8443, 13.8261]

Socket Python: Frame: 399 Pose: [114.0963134765625, 80.53250885009766, 184.47508239746094, -168.46871948242188, 23.844257354736328, 13.826091766357422]

NicerNewerCar commented 1 year ago

@amymmorton Are you calling setFrame before you get the pose/ncc?

That seems to solve the issue:

from PyAutoscoper.connect import AutoscoperConnection
import numpy as np
s = AutoscoperConnection()
# load trial, filters, and tracking data
nccData = np.zeros((400,2))
for i in range(400):
    s.setFrame(i)
    pose = s.getPose(1,i)
    ncc = s.getNCC(1,pose)
    nccData[i] = ncc

Set Frame

amymmorton commented 1 year ago

I was not previously using setFrame, I'm testing ncc output in matlab now

On Wed, Mar 1, 2023 at 10:16 AM Anthony Lombardi @.***> wrote:

@amymmorton https://github.com/amymmorton Are you calling setFrame before you get the pose/ncc?

That seems to solve the issue:

for i in range(400): s.setFrame(i) pose = s.getPose(1,i) ncc = s.getNCC(1,pose) nccData[i] = ncc

[image: Set Frame] https://user-images.githubusercontent.com/30351234/222181897-f535aa0d-2d4c-45b2-8a66-47e1f868cce2.png

— Reply to this email directly, view it on GitHub https://github.com/BrownBiomechanics/Autoscoper/issues/82#issuecomment-1450321731, or unsubscribe https://github.com/notifications/unsubscribe-auth/APUUPFWWRRJAQDUOVVROEF3WZ5R6VANCNFSM6AAAAAAUOAW344 . You are receiving this because you were mentioned.Message ID: @.***>

--

Amy Morton, MSc

Sr. Research Engineer, Bioengineering Lab, Department of Orthopaedic Research The Warren Alpert Medical School of Brown University and Rhode Island Hospital 1 Hoppin Street, CORO West, Ste. 404, Providence, RI 02903

amymmorton commented 1 year ago

Yes- that was the issue- Thank-you for catching Anthony! [image: image.png]

On Wed, Mar 1, 2023 at 11:58 AM Morton, Amy @.***> wrote:

I was not previously using setFrame, I'm testing ncc output in matlab now

On Wed, Mar 1, 2023 at 10:16 AM Anthony Lombardi @.***> wrote:

@amymmorton https://github.com/amymmorton Are you calling setFrame before you get the pose/ncc?

That seems to solve the issue:

for i in range(400): s.setFrame(i) pose = s.getPose(1,i) ncc = s.getNCC(1,pose) nccData[i] = ncc

[image: Set Frame] https://user-images.githubusercontent.com/30351234/222181897-f535aa0d-2d4c-45b2-8a66-47e1f868cce2.png

— Reply to this email directly, view it on GitHub https://github.com/BrownBiomechanics/Autoscoper/issues/82#issuecomment-1450321731, or unsubscribe https://github.com/notifications/unsubscribe-auth/APUUPFWWRRJAQDUOVVROEF3WZ5R6VANCNFSM6AAAAAAUOAW344 . You are receiving this because you were mentioned.Message ID: @.***>

--

Amy Morton, MSc

Sr. Research Engineer, Bioengineering Lab, Department of Orthopaedic Research The Warren Alpert Medical School of Brown University and Rhode Island Hospital 1 Hoppin Street, CORO West, Ste. 404, Providence, RI 02903

--

Amy Morton, MSc

Sr. Research Engineer, Bioengineering Lab, Department of Orthopaedic Research The Warren Alpert Medical School of Brown University and Rhode Island Hospital 1 Hoppin Street, CORO West, Ste. 404, Providence, RI 02903