search

ShuweiShao / AF-SfMLearner

[MedIA2022 & ICRA2021] Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescue
MIT License
101 stars 15 forks source link

Depth releated issuses #1

Closed Fannzi closed 2 years ago

Fannzi commented 2 years ago

Hi , thanks for the great work! I am a student at THU and I am also conducting some research in endoscopic depth perception. Got three questions regarding your work, glad if you could provide some insights.

1)You have used SCARED dataset posted by MICCA endovision challenge 2019, I noticed that the original depth images are of point clouds and in tiff format. I had some probs converting the depth into 2D gray scale depth images, but I failed to find the depth images processing part in your code, so I wonder if you could share some insights with me regarding the depth images preprocessing.

2)As stated, three datasets were used in your work. When evaluating the depth results of your depth network, how did you rescale your depth output to real distance space? You mentioned that the depth range for SCARED dataset was 0-150mm, how about the other two datasets?

3)The overall network contains pose-estimation and depth-estimation, so is the depth estimation part rather separately trained(as I have understood), or is it trained unitedly with the pose-estimation part's loss?

Thanks again for your work and looking forward to your reply!

ShuweiShao commented 2 years ago

Hi, thanks for the questions!

1) You could share me the detailed problems, and then I can make more targeted suggestions.

2) For the depth scaling, you can refer to the source code in evaluate_depth.py (lines 206-208); 1-150mm on the Hamlyn dataset and 1-180mm on the SERV-CT dataset. The depth ranges for three datasets are mentioned in the performance metrics of experimental settings.

3) The DepthNet and PoseNet are trained jointly duing the training phase, and evaluated separately at inference.

Fannzi commented 2 years ago

Hi, thanks for the quick reply.

Just one more question regarding the EndoVis dataset. I found that in your "export_gt_depth.py" code, you read the original .tiff images given. But I failed to read the depth. Is there something I missed? image

As I am training a depth network in png images mode, I wish to read the EndoVis dataset in gray-scale depth images mode. Glad if you can provide some help.

Thanks again!

ShuweiShao commented 2 years ago

Hi, it seems that the depth values are nan. You may check the source tiff file or the version of cv2. The version of opencv-contrib-python used in this work is 3.4.2.17. The code of saving depth in gray-scale image mode can be found in https://github.com/cleinc/bts/blob/master/pytorch/bts_test.py.

Fannzi commented 2 years ago

Many thanks! I will look into the cleinc/bts repository.

Fannzi commented 2 years ago

Hi,

Just one more question. The official dataset link to Hamlyn is invalid, wonder if you can share the Hamlyn dataset and the Endovis dataset with the png or jpeg format gray-scaled depth images?

Many thanks!!

ShuweiShao commented 2 years ago

Sorry for the late reply! I can share you the Hamlyn. For the Endovis dataset, this is not allowed by the collectors.

Fannzi commented 2 years ago

Hi, that would be wonderful! Thanks for the great help. My email is mmfan0010@163.com, or any other sharing means that is convenient to you is ok.

Gaoqinghong91 commented 2 years ago

Hi, could you share the Hamlyn datasets with me? My email is qgao@bournemouth.ac.uk. Thank you so much.

zzzh1y commented 1 year ago

Hi, I could you share the Hamlyn datasets with me as well? Thanks a lot! My email is zhouluyana@163.com

ZiWuAHaHa commented 1 month ago

Hi, could you share the Hamlyn datasets with me? My email is aze1596102@gmail.com. Thank you so much.