Training with TUM

[zzkslam]

Thanks for your excellent job.I have re-trianed your hfnet model with TUM dataset(picture size:640*480).But when I ran the _demo.ipynb_with the re-trained model,I got the error like this: Thanks in advance and looking forward your replys.

[zzkslam]

When I re-trianed your hfnet model with BDD dataset and I ran the _demo.ipynb_with the re-trained model, I got the same error. I think there may be some mistakes in my training process and my hfnet_train_distill.yaml is like this: hfnet_train_distill.txt

[sarlinpe]

Does it work with the original HF-Net work? Could you compare the two models?

[zzkslam]

Yes,the original model you provided works.Then I re-trianed your hfnet model with BDD dataset for 500 iterations and it didn't work.Can you tell me how to compare the two models?

[zzkslam]

I try to search in the saved graph.pbtxt and cannot find node named pred/simple_nms or pred/top_k_keypoints.The graph.pbtxt is like this:

graph.txt

[Gpetrak]

Hi, I have exactly the same problem after re-training the model. The interesting thing is that my hfnet_train_distill.yaml is quite similar with @zzkslam 's config file. Instead of training the hfnet based on mobilenetv2 weights, I re-train it with the hf-net weights that you have published:

`data:
name: 'distillation'
image_dirs: ['self_driving/images/images1', 'self_driving/images/images2']
load_targets: True
targets:
    - dir: 'global_descriptors'
      keys: ['global_descriptor']
    - dir: 'superpoint_predictions'
      keys: ['local_descriptor_map', 'dense_scores']
validation_size: 192
truncate: [185000, null, null]
preprocessing:
    resize: [480, 640]
    grayscale: true
augmentation:
    photometric:
        enable: true
        primitives: [
            'random_brightness', 'random_contrast',
            'additive_gaussian_noise', 'additive_shade', 'motion_blur']
        params:
            random_brightness: {delta_range: [-30., 40.]}
            random_contrast: {strength_range: [0.3, 1.2]}
            additive_gaussian_noise: {stddev_range: [0, 10]}
            additive_speckle_noise: {prob_range: [0, 0.0035]}
            additive_shade:
                transparency_range: [-0.5, 0.5]
                kernel_size_range: [100, 150]
                prob: 0.5
            motion_blur: {kernel_size: [13, 21], prob: 0.1}
    homographic:
        enable: false
        params:
            translation: true
            rotation: true
            scaling: true
            perspective: true
            scaling_amplitude: 0.2
            perspective_amplitude_x: 0.3
            perspective_amplitude_y: 0.3
            patch_ratio: 0.9
            max_angle: 0.3
            allow_artifacts: true
        valid_border_margin: 9
model:
name: 'hf_net'
image_channels: 1
depth_multiplier: 0.75
dropout_keep_prob: null
global:
    intermediate_proj: 0
    dimensionality_reduction: 4096
    n_clusters: 32
local:
    descriptor_dim: 256
#loss_weights: {local_desc: 1, global_desc: 1, detector: 1}
loss_weights: 'uncertainties'
train_backbone: true
batch_size: 16
eval_batch_size: 16
learning_rate: [0.001, 0.0001, 0.00001]
learning_rate_step: [60000, 80000]
weights: 'hf_net/model.ckpt-1000'
train_iter: 1000
validation_interval: 1000
save_interval: 5000
keep_checkpoints: 100
`

As I observed in Tensorboard, the layers "simple_nms" and "top_k_keypoitns" do not exist in the re-trained model:

Any thoughts ?

[Gpetrak]

Finally, I figured out to solve the issue. The reason for the lack of simple_nms and top_k_keypoints layers and their corresponding tensors was two lines that were missing in the config file ([hfnet_train_distill.yaml). When I changed the following piece of code:

local:
    descriptor_dim: 256

To

local:
    descriptor_dim: 256
    detector_threshold: 0.005
    nms_radius: 4
    num_keypoints: 1000

The problem solved :-)