Hello, I'm training a network with a data set of brain CT images to be able to then run a segmentation to identify stroke damage. For the most part I proceed similarly to the abdominal CT experiment, but with just one organ and 2 classes (normal regions and ROI).
After several tests and config modifications, I finally got output from a training (net_segment train...), but when I try to use this model.ckpt files to run a segmentation (net_segment inference...), it won't finish due to this issue:
CRITICAL: niftynet: checkpoint ~/models/model.ckpt-300 not found or variables to restore do not match the current application graph
...
NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
Key DenseVNet/batch_norm/DenseVNet/batch_norm/moving_mean/biased not found in checkpoint
[[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
Any idea why this is happening or how to configure the segmentation to check how the training results work?
Hello, I'm training a network with a data set of brain CT images to be able to then run a segmentation to identify stroke damage. For the most part I proceed similarly to the abdominal CT experiment, but with just one organ and 2 classes (normal regions and ROI). After several tests and config modifications, I finally got output from a training (net_segment train...), but when I try to use this model.ckpt files to run a segmentation (net_segment inference...), it won't finish due to this issue:
CRITICAL: niftynet: checkpoint ~/models/model.ckpt-300 not found or variables to restore do not match the current application graph ... NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error: Key DenseVNet/batch_norm/DenseVNet/batch_norm/moving_mean/biased not found in checkpoint [[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
Any idea why this is happening or how to configure the segmentation to check how the training results work?