search

dylan-campbell / bpnpnet

Solving the Blind Perspective-n-Point Problem End-To-End With Robust Differentiable Geometric Optimization
MIT License
74 stars 15 forks source link

Request for data preprocessing code #6

Closed SergioRAgostinho closed 3 years ago

SergioRAgostinho commented 3 years ago

Hey Dylan,

I'm trying to replicate your data generation setup on MegaDepth. I was crawling through the code that handles the dataset and noticed that the "unprocessed" version of code loads a number of pickle files. These files are not part of the native MegaDepth dataset so there's an intermediate stage missing here.

I wanted to ask if you could push these preprocessing scripts to this repository? We had a brief email exchange a couple of months ago that added some missing information. Here's what I know so far.

the splits are based on sequence numbers (randomly selected landmarks with no overlap between testing and training) For training, 1000 2D-3D correspondences were sampled randomly from each image--point set pair for testing we used all available points. No priority was given to images, resulting in a fairly imbalanced training set (ranging from 4 correspondences to 14742)

Training IDs of SfM models:

['0151_0_SIFT.pkl', '0306_0_SIFT.pkl', '0141_0_SIFT.pkl', '0183_0_SIFT.pkl', '0176_0_SIFT.pkl', '0176_1_SIFT.pkl', '0176_2_SIFT.pkl', '0095_0_SIFT.pkl', '0095_1_SIFT.pkl', '0095_2_SIFT.pkl', '0020_0_SIFT.pkl', '0020_1_SIFT.pkl', '0105_0_SIFT.pkl', '0004_0_SIFT.pkl', '0004_1_SIFT.pkl', '0004_2_SIFT.pkl', '0130_0_SIFT.pkl', '0130_1_SIFT.pkl', '0130_2_SIFT.pkl', '0078_0_SIFT.pkl', '0143_0_SIFT.pkl', '0019_0_SIFT.pkl', '0019_1_SIFT.pkl', '0107_0_SIFT.pkl', '0033_0_SIFT.pkl', '0017_0_SIFT.pkl', '0021_0_SIFT.pkl', '0046_0_SIFT.pkl', '0046_1_SIFT.pkl', '0046_2_SIFT.pkl', '0057_0_SIFT.pkl']

Testing IDs of SfM models:

['0104_0_SIFT.pkl', '0104_1_SIFT.pkl', '0026_0_SIFT.pkl', '0186_0_SIFT.pkl', '0067_0_SIFT.pkl', '0277_0_SIFT.pkl', '0277_1_SIFT.pkl', '0290_0_SIFT.pkl', '0290_1_SIFT.pkl', '0100_0_SIFT.pkl', '0022_0_SIFT.pkl', '0102_0_SIFT.pkl']

These IDs are selected randomly, and the name is under format: modelID_subModelID_SIFT.pkl.

I can probably recreate the process, but it would be always best to have access to the code that implemented the described logic.

Thank you for releasing the code

dylan-campbell commented 3 years ago

Hi Sergio,

I'll contact my co-author who looked after the pre-processing and get back to you shortly.

Cheers, Dylan

dylan-campbell commented 3 years ago

Hi @SergioRAgostinho, I don't have the pre-processing code yet, but I can outline it for you:

  1. read out the SfM model from the dataset
  2. go through each image in the model
  3. read its keypoints, rotation matrix, translation vector, intrinsic matrix, distortion parameters
  4. undistort the coordinates of keypoints
  5. for each keypoint, fetch its visible 3D index and save the 3D coordinates

Hope this helps!

SergioRAgostinho commented 3 years ago

Thank you.