RGB2Depth - Githubissues

fzd9752 commented 6 years ago

目的：验证现有模型预测深度的可靠性，为是否进一步改良模型提供依据。

[x] Quantitative Evaluation, 量化方法，与其他方式对比重现已有模型，根据评估误差横向评估需要输出图像达到256x256

2.- [ ] Deeper Depth Prediction with Fully Convolutional Residual Networks upsampling to 640x480 https://arxiv.org/pdf/1606.00373.pdf https://github.com/iro-cp/FCRN-DepthPrediction 代码不完整

4.Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields https://arxiv.org/pdf/1502.07411.pdf

6.Single-Image Depth Perception in the Wild https://arxiv.org/pdf/1604.03901.pdf
https://github.com/wfchen-umich/relative_depth 7.Multi-Scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation https://arxiv.org/pdf/1704.02157.pdf https://github.com/danxuhk/ContinuousCRF-CNN 选用

[ ] http://perso.ensta-paristech.fr/%7Epinard/depthnet/ 1/4 downsampling，缺失训练代码，可以预测 https://hal.archives-ouvertes.fr/hal-01587652/document https://github.com/ClementPinard/DepthNet
[ ] Depth Map Prediction from a Single Image using a Multi-Scale Deep Network 输出74x75 ,提供评估方法 https://www.cs.nyu.edu/~deigen/depth/ https://github.com/MasazI/cnn_depth_tensorflow 不完整 https://github.com/janivanecky/Depth-Estimation caffe
[x] Qualitative Evaluation，质化方法，点云图像化展示 depth to （x,y,z)点云array 快速转换：https://codereview.stackexchange.com/questions/79032/generating-a-3d-point-cloud 点云可视化 https://github.com/daavoo/pyntcloud https://github.com/createamind/busyplan/blob/master/zhangwei/point_cloud.ipynb

fzd9752 commented 6 years ago

深度图point cloud可视化：

[x] 深度 - > 点云
- [x] 点云转换python代码初步完成
- [x] 图片点云转换确认
[x] matplotlib 可视化
[x] 模型重新训练：
- 裁剪：图像等比例cropp3张数据图，最大限度保证源环境
- 预处理训练集：61000张图片
- 下午开始训练，预处理文件、训练数据、训练配置放在sdc pytorch-cyclegan 下 23_10_train 文件夹内
[x] 色温显示深度方法
- 初步代码确认，展现细节
[x] 完善用可视化评测深度预测的方法
- 多色彩2D展示
- 真实深度
- 生成深度
- 深度数值可视化统计
- 真实深度
- 生成深度
- 3D重建
- 真实深度
- 生成深度
[x] 等待新训练模型看效果

VKITTI 深度信息说明：

They correspond to the z coordinate of each pixel in camera coordinate space (not the distance to the camera optical center). camera intrinsic matrix (in pixels, constant, computed from our 1242x375 resolution and 29° fov):

          [[725,    0, 620.5],
 K =    [   0, 725, 187.0],
           [   0,     0,       1]]

点云转换原理 http://www.cnblogs.com/gaoxiang12/p/4652478.html 点云转换python https://codereview.stackexchange.com/questions/79032/generating-a-3d-point-cloud RVIZ - http://gazebosim.org/tutorials?tut=drcsim_visualization&cat= @waxz 提供的pyntcloud notebook中呈现3D效果

waxz commented 6 years ago

优化模型的方法 1、深度分割联合预测 http://users.eecs.northwestern.edu/~xsh835/assets/cvpr2015_depth.pdf 2.Single RGB Image Depth and Certainty Estimation via Deep Network and Dropout 3.深度图超分辨由于深度图一般的分辨率比输入图像小很多，需要采用超分辨率的方法增大分辨率 https://arxiv.org/pdf/1605.09546v1.pdf

zdx3578 commented 6 years ago

result show https://mp.weixin.qq.com/s?__biz=MzA5MDMwMTIyNQ==&mid=2649292578&idx=1&sn=ee9c909ef5d8000c821df48f2b006a46&chksm=8811e964bf666072622f38346dcba98d373a546db1187fb6948abea45357adeaa6dcaee9f580&scene=21#wechat_redirect

createamind / busyplan

RGB2Depth #5