proj_psnr VS psnr_3d

[1999kevin]

Nice work! I have run your code on the chest_50 data. I find that your method can achieve 32 psnr_3d but 12.8 proj_psnr. The proj_psnr is pretty low. I also try to change the hashgrid encoding to frequency encoding, which achieves 23.9 psnr_3d and 30.63 proj_psnr.

I wonder why your method achieves superior psnr_3d but a pretty low proj_psnr since you use the proj_mse to optimize the model.

It would be appreciated if you can provide some explanation.

[Ruyi-Zha]

Thanks for your interest in our work. I reran the code with the chest_50 data and obtained 33.2 psnr_3d and 40.9 proj_psnr. Could you please check if you made any modifications? If not, could you provide more details, such as logs, so that I can identify the potential problem?

My logs:

[Start] exp: chest_50, net: Basic network
[EVAL] epoch: 0/1500, proj_mse: 0.00603, proj_psnr: 7.51, psnr_3d: 9.77, ssim_3d: 0.315
[EVAL] epoch: 250/1500, proj_mse: 7.52e-07, proj_psnr: 37.3, psnr_3d: 30.7, ssim_3d: 0.945
[EVAL] epoch: 500/1500, proj_mse: 4.22e-07, proj_psnr: 38.5, psnr_3d: 32.4, ssim_3d: 0.959
[SAVE] epoch: 500/1500, path: ./logs/chest_50/ckpt.tar                          
[EVAL] epoch: 750/1500, proj_mse: 3.78e-07, proj_psnr: 40, psnr_3d: 32.8, ssim_3d: 0.962
[EVAL] epoch: 1000/1500, proj_mse: 5.44e-07, proj_psnr: 37, psnr_3d: 33, ssim_3d: 0.963
[SAVE] epoch: 1000/1500, path: ./logs/chest_50/ckpt.tar                         
[EVAL] epoch: 1250/1500, proj_mse: 4.17e-07, proj_psnr: 38.6, psnr_3d: 33.2, ssim_3d: 0.964
[EVAL] epoch: 1500/1500, proj_mse: 3.06e-07, proj_psnr: 40.9, psnr_3d: 33.2, ssim_3d: 0.964
[SAVE] epoch: 1500/1500, path: ./logs/chest_50/ckpt.tar                         
Training complete! See logs in ./logs/chest_50                        
epoch=1500/1500, loss=8.3e-08, lr=0.0001: : 75050it [10:37, 117.82it/s]

[1999kevin]

@Ruyi-Zha Thank you for your reply. I find that this is because I use the chest_50 data generated by myself from raw data with the generateData.py. I have generated twice and run the experiments twice. The proj_pnsr is still pretty low. See the logs here:

[Start] exp: chest_50_2, net: Basic network
epoch=0/1500, loss=0.00438, lr=0.001:   0%|          | 1/75000 [00:01<30:02:08,  1.44s/it][EVAL] epoch: 0/1500, proj_mse: 0.00756, proj_psnr: 7.04, psnr_3d: 9.64, ssim_3d: 0.314
epoch=250/1500, loss=9.73e-08, lr=0.001:  17%|█▋        | 12504/75000 [02:13<10:52, 95.83it/s][EVAL] epoch: 250/1500, proj_mse: 0.00017, proj_psnr: 12.2, psnr_3d: 31, ssim_3d: 0.947
epoch=500/1500, loss=6.2e-08, lr=0.001:  33%|███▎      | 25006/75000 [04:26<48:39, 17.12it/s] [EVAL] epoch: 500/1500, proj_mse: 0.000189, proj_psnr: 13.2, psnr_3d: 31.7, ssim_3d: 0.952
epoch=500/1500, loss=1.02e-07, lr=0.001:  33%|███▎      | 25050/75000 [04:27<18:22, 45.31it/s][SAVE] epoch: 500/1500, path: ./logs/chest_50_2/ckpt.tar
epoch=750/1500, loss=7.72e-08, lr=0.001:  50%|█████     | 37501/75000 [06:39<06:22, 98.04it/s][EVAL] epoch: 750/1500, proj_mse: 0.00021, proj_psnr: 11.4, psnr_3d: 31.9, ssim_3d: 0.953
epoch=1000/1500, loss=6.27e-08, lr=0.001:  67%|██████▋   | 50001/75000 [08:52<04:23, 94.92it/s][EVAL] epoch: 1000/1500, proj_mse: 0.000225, proj_psnr: 12, psnr_3d: 32, ssim_3d: 0.954
epoch=1000/1500, loss=7.11e-08, lr=0.001:  67%|██████▋   | 50050/75000 [08:52<09:07, 45.55it/s][SAVE] epoch: 1000/1500, path: ./logs/chest_50_2/ckpt.tar
epoch=1250/1500, loss=5.29e-08, lr=0.001:  83%|████████▎ | 62502/75000 [11:04<10:53, 19.12it/s][EVAL] epoch: 1250/1500, proj_mse: 0.000245, proj_psnr: 12, psnr_3d: 32, ssim_3d: 0.954
epoch=1500/1500, loss=5.86e-08, lr=0.0001: 100%|██████████| 75000/75000 [13:15<00:00, 95.46it/s][EVAL] epoch: 1500/1500, proj_mse: 0.000265, proj_psnr: 12, psnr_3d: 32, ssim_3d: 0.954
epoch=1500/1500, loss=5.15e-08, lr=0.0001: : 75050it [13:16, 45.19it/s][SAVE] epoch: 1500/1500, path: ./logs/chest_50_2/ckpt.tar
epoch=1500/1500, loss=5.15e-08, lr=0.0001: : 75050it [13:17, 94.10it/s]
Training complete! See logs in ./logs/chest_50_2

I also run the experiment on your example chest_50.pickle. The logs are shown here:

0%|          | 0/75000 [00:00<?, ?it/s][Start] exp: chest_50_2, net: Basic network
epoch=0/1500, loss=0.0051, lr=0.001:   0%|          | 1/75000 [00:01<33:15:57,  1.60s/it][EVAL] epoch: 0/1500, proj_mse: 0.00682, proj_psnr: 7.73, psnr_3d: 9.11, ssim_3d: 0.31
epoch=250/1500, loss=2.02e-07, lr=0.001:  17%|█▋        | 12501/75000 [02:14<10:51, 95.97it/s][EVAL] epoch: 250/1500, proj_mse: 3.68e-06, proj_psnr: 29.5, psnr_3d: 30.8, ssim_3d: 0.946
epoch=500/1500, loss=1.22e-07, lr=0.001:  33%|███▎      | 25001/75000 [04:26<08:06, 102.76it/s][EVAL] epoch: 500/1500, proj_mse: 1.89e-06, proj_psnr: 33.1, psnr_3d: 32.1, ssim_3d: 0.957
epoch=500/1500, loss=1.42e-07, lr=0.001:  33%|███▎      | 25050/75000 [04:26<19:10, 43.43it/s][SAVE] epoch: 500/1500, path: ./logs/chest_50_2/ckpt.tar
epoch=750/1500, loss=1.1e-07, lr=0.001:  50%|█████     | 37501/75000 [06:39<06:19, 98.88it/s] [EVAL] epoch: 750/1500, proj_mse: 1.8e-06, proj_psnr: 32.8, psnr_3d: 32.5, ssim_3d: 0.96
epoch=1000/1500, loss=5.75e-08, lr=0.001:  67%|██████▋   | 50005/75000 [08:52<04:16, 97.26it/s][EVAL] epoch: 1000/1500, proj_mse: 2.01e-06, proj_psnr: 32.8, psnr_3d: 32.7, ssim_3d: 0.961
epoch=1000/1500, loss=1.03e-07, lr=0.001:  67%|██████▋   | 50050/75000 [08:52<11:05, 37.47it/s][SAVE] epoch: 1000/1500, path: ./logs/chest_50_2/ckpt.tar
epoch=1250/1500, loss=9.02e-08, lr=0.001:  83%|████████▎ | 62501/75000 [11:04<02:11, 95.31it/s][EVAL] epoch: 1250/1500, proj_mse: 1.98e-06, proj_psnr: 33.1, psnr_3d: 32.9, ssim_3d: 0.962
epoch=1500/1500, loss=5.66e-08, lr=0.0001: : 75008it [13:16, 18.14it/s][EVAL] epoch: 1500/1500, proj_mse: 2.56e-06, proj_psnr: 30.3, psnr_3d: 33, ssim_3d: 0.962
epoch=1500/1500, loss=8.92e-08, lr=0.0001: : 75050it [13:17, 47.03it/s][SAVE] epoch: 1500/1500, path: ./logs/chest_50_2/ckpt.tar
epoch=1500/1500, loss=8.92e-08, lr=0.0001: : 75050it [13:18, 94.01it/s]
Training complete! See logs in ./logs/chest_50_2

No modification in chest_50.yaml except the name:

exp:
  expname: chest_50_2
  expdir: ./logs/
  datadir: ./data/chest_50_2.pickle
network:
  net_type: mlp
  num_layers: 4
  hidden_dim: 32
  skips: [2]
  out_dim: 1
  last_activation: sigmoid
  bound: 0.3
encoder:
  encoding: hashgrid
  input_dim: 3
  num_levels: 16
  level_dim: 2
  base_resolution: 16
  log2_hashmap_size: 19
render:
  n_samples: 192
  n_fine: 0
  perturb: True
  raw_noise_std: 0.
  netchunk: 409600
train:
  epoch: 1500
  n_batch: 1
  n_rays: 1024
  lrate: 0.001
  lrate_gamma: 0.1
  lrate_step: 1500
  resume: False
log:
  i_eval: 250  # Epoch for evaluation
  i_save: 500  # Epoch for saving

Also no modification for the config.yml for generating chest data:

# Parameters to setup CT simulation.

# dataset
numTrain: 50
numVal: 50

# System configuration
DSD: 1500.0  # Distance Source Detector      (mm)
DSO: 1000.0  # Distance Source Origin        (mm)

# Detector parameters
nDetector: # Number of pixels              (px)
- 256 
- 256 
dDetector: # Size of each pixel            (mm)
- 1.5
- 1.5

# Image parameters
nVoxel: # Number of voxels              (vx)
- 128
- 128
- 128 
dVoxel: # size of each voxel            (mm)
- 1.0
- 1.0
- 1.0

# Offsets
offOrigin: # Offset of image from origin   (mm)
- 0 # x direction
- 0 # y direction
- 0 # z direction
offDetector: # Offset of Detector (only in two direction)            (mm)
- 0 # u direction
- 0 # v direction

# Auxiliary
accuracy: 0.5 # Accuracy of FWD proj          (vx/sample)

# Mode
mode: cone # X-ray source mode parallel/cone
filter: null

# Angles
totalAngle: 180.0 # Total angle  (degree)
startAngle: 0.0 # Start Angle (degree)
randomAngle: False

# CT
convert: False
rescale_slope: 1.0
rescale_intercept: 0.0
normalize: True

# Noise
noise: 0

[Ruyi-Zha]

Thank you for your reply. I identified the issue: I mistakenly set dDetector in chest/config.yml to 1.5 instead of 1.0. This caused some rays to be outside the predefined space, leading to artifacts in the rendered 2D projections and low psnr_proj values. I have updated the repository and data. You should now be able to get the expected results. Thanks for your help.

dDetector=1.5

dDetector=1.0

[1999kevin]

Thank you for your reply. Nice work!