How to train in MegaDepth without D2-Net data

[QiuhangLiu]

Hi authors, thanks for your great works! I see the F&Q, since we cannot reach D2_net pre-processing data anymore, you suggest us to use MegaDepth directly. May I ask how to use it for training? Like when we build the symlink, how could the training be done? Could you please update a new guidance for training?

ln -sv /path/to/megadepth/phoenix /path/to/LoFTR/data/megadepth/train ln -sv /path/to/megadepth/phoenix /path/to/LoFTR/data/megadepth/test

ln -s /path/to/megadepth_indices/* /path/to/LoFTR/data/megadepth/index

Are the above codes correct? Or how could we use the data?

[QiuhangLiu]

Hi authors, thanks for your great works! I see the F&Q, since we cannot reach D2_net pre-processing data anymore, you suggest us to use MegaDepth directly. May I ask how to use it for training? Like when we build the symlink, how could the training be done? Could you please update a new guidance for training?

ln -sv /path/to/megadepth/phoenix /path/to/LoFTR/data/megadepth/train ln -sv /path/to/megadepth/phoenix /path/to/LoFTR/data/megadepth/test

ln -s /path/to/megadepth_indices/* /path/to/LoFTR/data/megadepth/index

Are the above codes correct? Or how could we use the data?

Or shall I download that Megadepth sfm data to replace the D2Net Data?

[nonathecoda]

You need to change the NPZ files in index. Before training, I ran this script:

import numpy as np
from numpy import load
import os

#change scene_info_0
directory = 'PATH_TO_LOFTR/data/megadepth/index/scene_info_0.1_0.7'

for filename in os.listdir(directory):
    f_npz = os.path.join(directory,filename)
    data = load(f_npz, allow_pickle = True)
    for count, image_path in enumerate(data['image_paths']):
        if image_path is not None:
            if 'Undistorted_SfM' in image_path:
                data['image_paths'][count] = data['depth_paths'][count].replace('depths', 'imgs').replace('h5', 'jpg')

    data['pair_infos'] = np.asarray(data['pair_infos'], dtype=object)
    new_file = 'PATH_TO_LOFTR/data/megadepth/index/scene_info_0.1_0.7_no_sfm/' + filename
    np.savez(new_file, **data)
    print("Saved to ", new_file)

#change scene_info_val_1500
directory = 'PATH_TO_LOFTR/data/megadepth/index/scene_info_val_1500'

for filename in os.listdir(directory):
    f_npz = os.path.join(directory,filename)
    data = load(f_npz, allow_pickle = True)
    for count, image_path in enumerate(data['image_paths']):
        if image_path is not None:
            if 'Undistorted_SfM' in image_path:
                data['image_paths'][count] = data['depth_paths'][count].replace('depths', 'imgs').replace('h5', 'jpg')

    data['pair_infos'] = np.asarray(data['pair_infos'], dtype=object)
    new_file = 'PATH_TO_LOFTR/data/megadepth/index/scene_info_val_1500_no_sfm/' + filename
    np.savez(new_file, **data)
    print("Saved to ", new_file)`

Then also run the following script to make sure all images have the ending 'jpg' (there are some hidden JPG and pngs in the dataset :) )

import os
from PIL import Image

root_directory = '/PATH_TO_DATASET/phoenix/S6/zl548/MegaDepth_v1'

for folder in os.listdir(root_directory):
    four_digit_directory = os.path.join(root_directory,folder)
    for dense_folder in os.listdir(four_digit_directory):
        image_directory =  os.path.join(four_digit_directory,dense_folder,'imgs')
        for image in os.listdir(image_directory):
            if 'JPG' in image:
                new_name = image.replace('JPG', 'jpg')
                old_path = os.path.join(image_directory, image)
                new_path = os.path.join(image_directory, new_name)
                os.rename(old_path, new_path)
            if 'png' in image:
                new_name = image.replace('png', 'jpg')
                old_path = os.path.join(image_directory, image)
                new_path = os.path.join(image_directory, new_name)
                png_img = Image.open(old_path)
                png_img.save(new_path)

Then, I changed the following lines in LoFTR/configs/data/megadepth_trainval_640.py:

cfg.DATASET.TRAIN_NPZ_ROOT = f"{TRAIN_BASE_PATH}/scene_info_0.1_0.7_no_sfm"
cfg.DATASET.VAL_NPZ_ROOT = cfg.DATASET.TEST_NPZ_ROOT = f"{TEST_BASE_PATH}/scene_info_val_1500_no_sfm"

Lastly you need to typecast to dict in line 47 in LoFTR/src/datasets/megadepth.py:

self.intrinsics = dict(np.load(intrinsic_path))

Hope everything works!

[EdithYoung]

hey i'm also tring to reproduce the training process, and i'm a Chinese student too, can i have your contact information so that i can discuss some questions abouit this project with you?

[QiuhangLiu]

hey i'm also tring to reproduce the training process, and i'm a Chinese student too, can i have your contact information so that i can discuss some questions abouit this project with you?

Sure, wechat number 1065043641 :)

[Master-cai]

Lastly you need to typecast to dict in line 47 in LoFTR/src/datasets/megadepth.py:

self.intrinsics = dict(np.load(intrinsic_path))

Hope everything works!

Thanks for your careful guide! but the last step may be a bit of confusion. the code in line 47 in megadepth.py should be

self.scene_info = dict(np.load(npz_path, allow_pickle=True))

[RunyuZhu]

hey i'm also tring to reproduce the training process, and i'm a Chinese student too, can i have your contact information so that i can discuss some questions abouit this project with you?

Sure, wechat number 1065043641 :)

sorry for disturbing you, im a Chinese student too, i wonder if i can also add your wechat for consulting some problem about loftr's train. thanks zry

[jxhoh]

intrinsics I downloaded megadepth data. Can I use the 200g one directly? Or do I need to download the 900g one with sfm

[jxhoh]

self.scene_info = dict(np.load(npz_path, allow_pickle=True))

thanks，Use this code, replace your last sentence, I've trained it, thank you

[panjiadipanjiadi]

self.scene_info = dict（np.load（npz_path， allow_pickle=True））

谢谢，用这段代码，把你最后一句话换掉，我已经训练过了，谢谢

你好，问下你最后使用200g进行训练的吗，最后结果怎么样

[Inverse-function]

You need to change the NPZ files in index. Before training, I ran this script:

import numpy as np
from numpy import load
import os

#change scene_info_0
directory = 'PATH_TO_LOFTR/data/megadepth/index/scene_info_0.1_0.7'

for filename in os.listdir(directory):
    f_npz = os.path.join(directory,filename)
    data = load(f_npz, allow_pickle = True)
    for count, image_path in enumerate(data['image_paths']):
        if image_path is not None:
            if 'Undistorted_SfM' in image_path:
                data['image_paths'][count] = data['depth_paths'][count].replace('depths', 'imgs').replace('h5', 'jpg')

    data['pair_infos'] = np.asarray(data['pair_infos'], dtype=object)
    new_file = 'PATH_TO_LOFTR/data/megadepth/index/scene_info_0.1_0.7_no_sfm/' + filename
    np.savez(new_file, **data)
    print("Saved to ", new_file)

#change scene_info_val_1500
directory = 'PATH_TO_LOFTR/data/megadepth/index/scene_info_val_1500'

for filename in os.listdir(directory):
    f_npz = os.path.join(directory,filename)
    data = load(f_npz, allow_pickle = True)
    for count, image_path in enumerate(data['image_paths']):
        if image_path is not None:
            if 'Undistorted_SfM' in image_path:
                data['image_paths'][count] = data['depth_paths'][count].replace('depths', 'imgs').replace('h5', 'jpg')

    data['pair_infos'] = np.asarray(data['pair_infos'], dtype=object)
    new_file = 'PATH_TO_LOFTR/data/megadepth/index/scene_info_val_1500_no_sfm/' + filename
    np.savez(new_file, **data)
    print("Saved to ", new_file)`

Then also run the following script to make sure all images have the ending 'jpg' (there are some hidden JPG and pngs in the dataset :) )

import os
from PIL import Image

root_directory = '/PATH_TO_DATASET/phoenix/S6/zl548/MegaDepth_v1'

for folder in os.listdir(root_directory):
    four_digit_directory = os.path.join(root_directory,folder)
    for dense_folder in os.listdir(four_digit_directory):
        image_directory =  os.path.join(four_digit_directory,dense_folder,'imgs')
        for image in os.listdir(image_directory):
            if 'JPG' in image:
                new_name = image.replace('JPG', 'jpg')
                old_path = os.path.join(image_directory, image)
                new_path = os.path.join(image_directory, new_name)
                os.rename(old_path, new_path)
            if 'png' in image:
                new_name = image.replace('png', 'jpg')
                old_path = os.path.join(image_directory, image)
                new_path = os.path.join(image_directory, new_name)
                png_img = Image.open(old_path)
                png_img.save(new_path)

Then, I changed the following lines in LoFTR/configs/data/megadepth_trainval_640.py:

cfg.DATASET.TRAIN_NPZ_ROOT = f"{TRAIN_BASE_PATH}/scene_info_0.1_0.7_no_sfm"
cfg.DATASET.VAL_NPZ_ROOT = cfg.DATASET.TEST_NPZ_ROOT = f"{TEST_BASE_PATH}/scene_info_val_1500_no_sfm"

Lastly you need to typecast to dict in line 47 in LoFTR/src/datasets/megadepth.py:

self.intrinsics = dict(np.load(intrinsic_path))

Hope everything works!

你好，在我运行第一个脚本之后，报出错误：没有scene_info_0.1_0.7_no_sfm这个文件，请问我应该怎么办

[kangpeilun]

self.scene_info = dict（np.load（npz_path， allow_pickle=True））

谢谢，用这段代码，把你最后一句话换掉，我已经训练过了，谢谢

你好，问下你最后使用200g进行训练的吗，最后结果怎么样

I use 200g Megadepth datasets for training successfully, but my GPU mem is 12G, so when I train LoFTR model arising out of memory.