Open seigeweapon opened 4 years ago
@seigeweapon, Hi seigeweapon, I also have the same problem. Have you solved it?
No. I'm still having problem. I put my code and result here, hopefully someone can point out what's wrong.
` add_path;
img_w = 1280;
img_h = 1024;
intrinsics = [
1075.72 0 636.877 0 1075.66 515.013 0 0 1
1075.94 0 629.693 0 1076.09 501.638 0 0 1
1074.61 0 630.692 0 1074.93 523.124 0 0 1
];
data_path = 'D:\data\matterport\17DRP5sb8fy';
scene_id = '00ebbf3782c64d74aaf7dd39cd561175';
sepImg = [];
rts = [
-0.73677 -0.438446 -0.514718 3.52064 -0.675941 0.496262 0.544819 -0.501891 0.0165615 0.749326 -0.661995 1.54282 0 0 0 1
-0.953749 0.211332 0.213781 3.50647 0.300171 0.631344 0.715054 -0.505191 0.0161444 0.746152 -0.665579 1.54289 0 0 0 1
-0.216901 0.653201 0.725453 3.49653 0.976122 0.136149 0.169259 -0.494569 0.0117899 0.744843 -0.667135 1.54292 0 0 0 1
0.736908 0.445283 0.508615 3.50076 0.675947 -0.494116 -0.546758 -0.480646 0.00785256 0.746707 -0.665106 1.54288 0 0 0 1
0.953849 -0.2045 -0.219891 3.51493 -0.300173 -0.629173 -0.716964 -0.477347 0.00826982 0.749881 -0.661521 1.54281 0 0 0 1
0.216975 -0.64635 -0.731542 3.52487 -0.976096 -0.133961 -0.171149 -0.48797 0.0126244 0.75119 -0.659965 1.54278 0 0 0 1
-0.729995 0.00798594 -0.683406 3.51324 -0.683354 0.00847387 0.730038 -0.493776 0.011621 0.999932 -0.000728289 1.57609 0 0 0 1
-0.956787 0.0123707 0.290529 3.50983 0.290574 0.00201265 0.956851 -0.49457 0.011252 0.999922 -0.00551992 1.5761 0 0 0 1
-0.226736 0.0089675 0.973915 3.50744 0.973932 -0.00501523 0.226786 -0.492015 0.00691798 0.999948 -0.00759623 1.57611 0 0 0 1
0.730089 0.00117961 0.683352 3.50846 0.683346 -0.00558174 -0.730073 -0.488665 0.00295295 0.999984 -0.00488087 1.5761 0 0 0 1
0.956842 -0.00320491 -0.290592 3.51187 -0.29059 0.000879651 -0.956847 -0.487872 0.00332209 0.999995 -8.92584e-05 1.57609 0 0 0 1
0.226765 0.000198574 -0.97395 3.51426 -0.97392 0.0079074 -0.226756 -0.490427 0.00765625 0.999969 0.00198689 1.57608 0 0 0 1
-0.728354 0.466698 -0.50169 3.52095 -0.68513 -0.485575 0.542967 -0.501795 0.00979302 0.739196 0.673419 1.60931 0 0 0 1
-0.957507 -0.186404 0.220078 3.50678 0.288256 -0.643483 0.709111 -0.505095 0.00943564 0.742417 0.669871 1.60938 0 0 0 1
-0.229106 -0.649703 0.724843 3.49683 0.973388 -0.156822 0.167099 -0.494473 0.00510657 0.743837 0.668342 1.60941 0 0 0 1
0.728431 -0.45989 0.507828 3.50106 0.685118 0.487735 -0.541043 -0.48055 0.00113495 0.742035 0.67036 1.60938 0 0 0 1
0.957545 0.193218 -0.213947 3.51523 -0.288278 0.645618 -0.707159 -0.477251 0.00149246 0.738813 0.673908 1.60931 0 0 0 1
0.229118 0.656498 -0.718691 3.52517 -0.973381 0.158939 -0.165128 -0.487873 0.00582165 0.737394 0.675438 1.60928 0 0 0 1
];
idx = 1;
for j = 1:3
intrinsic = transpose(reshape(intrinsics(j,:), [3,3]));
fx = intrinsic(1,1);
fy = intrinsic(2,2);
cx = intrinsic(1,3);
cy = intrinsic(2,3);
crop_x_start = 1;
crop_x_end = img_w;
crop_y_start = 1;
crop_y_end = img_h;
if cx * 2 <= img_w
crop_x_end = round(cx*2);
else
crop_x_start = round(1 + cx*2 - img_w);
end
if cy * 2 <= img_h
crop_y_end = round(cy*2);
else
crop_y_start = round(crop_y_start + cy*2 - img_h);
end
hfov = 2 * atan(((crop_x_end - crop_x_start + 1) / 2 / fx));
for i = 1:6
img = im2double(imread(fullfile(data_path, 'undistorted_color_images', ...
sprintf('%s_i%d_%d.jpg', scene_id, j-1, i-1))));
% get yaw roll pitch from rt
rt = transpose(reshape(rts((j-1)*6+i,:), [4,4]));
r = rt(1:3,1:3);
yrp = num2cell(rotm2eul(r));
[yaw roll pitch] = yrp{:};
pitch = pitch - 1.57;
fprintf("%f %f %f %f\n", yaw, roll, pitch, hfov);
% rotate image
img = imrotate(img, -roll * 180 / pi, 'bilinear', 'crop');
% crop image so that camera center is image center
img = img(crop_y_start:crop_y_end,crop_x_start:crop_x_end,:);
sepImg(idx).img = img;
sepImg(idx).vx = -yaw;
sepImg(idx).vy = pitch;
sepImg(idx).fov = hfov;
sepImg(idx).sz = size(sepImg(idx).img);
idx = idx + 1;
end
end
panocolor = combineViews( sepImg, 2048, 1024 );
figure; imshow(panocolor);
`
@seigeweapon @HalleyJiang Any update on this? I rendered the depth for skybox images using blender but it is still not perfectly aligned. Does anybody know the exact poses for the skybox images?
The authors even could not reply about the exact poses for the skybox images. We can just approximately align the depth to the panoramas from the skybox images. It's ok for panorama depth estimation. Please refer to our work on preprocessing Matterport3D(https://github.com/alibaba/UniFuse-Unidirectional-Fusion/blob/main/UniFuse/Matterport3D/README.md).
Hey Thanks for the answer. On the repo you sent, there is a function called stitch in stitching_Matterport3D.m
. There is no implementation of that function and matlab doesnt find it. Am I missing anything?
You need to download PanoBasic first. Please carefully read the README.
I did it. But there is not stitch function anywhere and raises an error
We missed some .m files. We have added them. Please try again.
Although the dataset is awesome, I find it very inconvenient not having a pixel-corresponding depth map in skybox form. It provides only color skybox. I tried using PanoBasic to stitch that out of the undistorted color/depth images, with camera extrinsics from undistorted_camera_parameters. But the result is poor. Then I tried using commercial stitching software like PTGUI, but only find it's hard because of very small overlapping area.
Anybody run into this like me? Help me please!