Closed godspeed1989 closed 5 years ago
Can you please check the PSMNet environment? Python2.7 PyTorch(0.4.0+) torchvision 0.2.0 (higher version may cause issues)
Hi, I am using python3 and pytorch 1.1.0. Should I retrain PSMNet? That's wired, is there any simpler solution. Thank you
The simpler solution is you install Python2.7, PyTorch(0.4.0+), torchvision 0.2. You can easily use anaconda to create a new environment.
Thanks.
Hi, Is this how it is supposed to be now ?
But, I notice the generated pseudo point cloud seems not similar to the ones in your papers
It should not look like it. How do you generate this point cloud? which code do you use?
I follow your README to generate point cloud. Firstly, generating disparity map
$ python ./psmnet/submission.py \
--loadmodel ./finetune_300.tar \
--datapath /mine/KITTI_DAT/training/ \
--save_path ./training_predict_disparity/
Then, converting disparity to point cloud
python ./preprocessing/generate_lidar.py \
--calib_path /mine/KITTI_DAT/training/calib \
--save_path ./training_predict_velodyne \
--disparity_dir ./training_predict_disparity/ \
--max_high 1
I notice the --disp_path
cmd option is incorrect, so i use --disparity_dir
instead
This is the 000003
file in KITTI training set
I also found the script generate_lidar.py
does not work properly!
I used frustum-pointnets, first feed the ground truth velodyne to it for testing, and it worked well. Then I use generate_lidar.py
to generate predicted_velodyne from ground disparity, and feed predicted_velodyne for testing, the results are very bad. So I think there should be some problem in generate_lidar.py
.
I follow your README to generate point cloud. Firstly, generating disparity map
$ python ./psmnet/submission.py \ --loadmodel ./finetune_300.tar \ --datapath /mine/KITTI_DAT/training/ \ --save_path ./training_predict_disparity/
Then, converting disparity to point cloud
python ./preprocessing/generate_lidar.py \ --calib_path /mine/KITTI_DAT/training/calib \ --save_path ./training_predict_velodyne \ --disparity_dir ./training_predict_disparity/ \ --max_high 1
I notice the
--disp_path
cmd option is incorrect, so i use--disparity_dir
instead This is the000003
file in KITTI training set
Hi godspeed1989,
I used the code in this repo to generate the point cloud. It looks correct. Not sure why your result is not right. So Can you send me the disparity file and bin file of 000003 to me? My email is yw763@cornell.edu . Thanks.
I also found the script
generate_lidar.py
does not work properly! I used frustum-pointnets, first feed the ground truth velodyne to it for testing, and it worked well. Then I usegenerate_lidar.py
to generate predicted_velodyne from ground disparity, and feed predicted_velodyne for testing, the results are very bad. So I think there should be some problem ingenerate_lidar.py
.
Hi zklgame,
Did you use the FPointNet trained on velodyne or on pseudo-lidar? Might you try this checkpoint https://drive.google.com/file/d/1qhCxw6uHqQ4SAkxIuBi-QCKqLmTGiNhP/view?usp=sharing, which is trained on pseudo-lidar.
Hi, mileyan.
I add --save_float
option to run python ./psmnet/submission.py
and generate disparity map stored in npy.
In the meanwhile, modified serveral lines in ./preprocessing/generate_lidar.py
to read .npy instead of .png.
The result looks better.
training/000000
training/000002
training/000003
But i not sure whether it is correct now. I notice the estimated depth is not so good at the distance.
Hi godspeed1989,
It looks great! I will change the save_float
to the default setting. Thanks for your feedback.
@godspeed1989 May I ask you what visualization tool you use for the point cloud (.bin)?
Thank you!
@swdev1202
I am using pyqtgraph
to visual point cloud.
You can install it use pip
from pyqtgraph.Qt import QtCore, QtGui
import pyqtgraph.opengl as gl
class plot3d(object):
def __init__(self, title='null'):
#
self.glview = gl.GLViewWidget()
coord = gl.GLAxisItem()
coord.setSize(1,1,1)
#self.glview.addItem(coord)
self.glview.setMinimumSize(QtCore.QSize(600, 500))
self.glview.pan(1, 0, 0)
self.glview.setCameraPosition(azimuth=180)
self.glview.setCameraPosition(elevation=0)
self.glview.setCameraPosition(distance=5)
self.items = []
#
self.view = QtGui.QWidget()
self.view.window().setWindowTitle(title)
hlayout = QtGui.QHBoxLayout()
snap_btn = QtGui.QPushButton('&Snap')
def take_snap():
qimg = self.glview.readQImage()
qimg.save('1.jpg')
snap_btn.clicked.connect(take_snap)
hlayout.addWidget(snap_btn)
hlayout.addStretch()
layout = QtGui.QVBoxLayout()
#
layout.addLayout(hlayout)
layout.addWidget(self.glview)
self.view.setLayout(layout)
def add_item(self, item):
self.glview.addItem(item)
self.items.append(item)
def clear(self):
for it in self.items:
self.glview.removeItem(it)
self.items.clear()
def add_points(self, points, colors):
points_item = gl.GLScatterPlotItem(pos=points, size=1.5, color=colors)
self.add_item(points_item)
def add_line(self, p1, p2, color, width=3):
lines = np.array([[p1[0], p1[1], p1[2]],
[p2[0], p2[1], p2[2]]])
lines_item = gl.GLLinePlotItem(pos=lines, mode='lines',
color=color, width=width, antialias=True)
self.add_item(lines_item)
def plot_bbox_mesh(self, gt_boxes3d, color=(0,1,0,1)):
b = gt_boxes3d
for k in range(0,4):
i,j=k,(k+1)%4
self.add_line([b[i,0],b[i,1],b[i,2]], [b[j,0],b[j,1],b[j,2]], color)
i,j=k+4,(k+1)%4 + 4
self.add_line([b[i,0],b[i,1],b[i,2]], [b[j,0],b[j,1],b[j,2]], color)
i,j=k,k+4
self.add_line([b[i,0],b[i,1],b[i,2]], [b[j,0],b[j,1],b[j,2]], color)
def value_to_rgb(pc_inte):
minimum, maximum = np.min(pc_inte), np.max(pc_inte)
ratio = (pc_inte - minimum + 0.1) / (maximum - minimum + 0.1)
r = (np.maximum((1 - ratio), 0))
b = (np.maximum((ratio - 1), 0))
g = 1 - b - r
return np.stack([r, g, b]).transpose()
def view_points_cloud(pc=None):
app = QtGui.QApplication([])
glview = plot3d()
if pc is None:
pc = np.random.rand(1024, 3)
pc_color = np.ones([pc.shape[0], 4])
glview.add_points(pc, pc_color)
glview.view.show()
return app.exec()
@godspeed1989 godspeed1989 Hi,How does this code apply and where do I need to put it?
@swdev1202 I am using
pyqtgraph
to visual point cloud. You can install it use pipfrom pyqtgraph.Qt import QtCore, QtGui import pyqtgraph.opengl as gl class plot3d(object): def __init__(self, title='null'): # self.glview = gl.GLViewWidget() coord = gl.GLAxisItem() coord.setSize(1,1,1) #self.glview.addItem(coord) self.glview.setMinimumSize(QtCore.QSize(600, 500)) self.glview.pan(1, 0, 0) self.glview.setCameraPosition(azimuth=180) self.glview.setCameraPosition(elevation=0) self.glview.setCameraPosition(distance=5) self.items = [] # self.view = QtGui.QWidget() self.view.window().setWindowTitle(title) hlayout = QtGui.QHBoxLayout() snap_btn = QtGui.QPushButton('&Snap') def take_snap(): qimg = self.glview.readQImage() qimg.save('1.jpg') snap_btn.clicked.connect(take_snap) hlayout.addWidget(snap_btn) hlayout.addStretch() layout = QtGui.QVBoxLayout() # layout.addLayout(hlayout) layout.addWidget(self.glview) self.view.setLayout(layout) def add_item(self, item): self.glview.addItem(item) self.items.append(item) def clear(self): for it in self.items: self.glview.removeItem(it) self.items.clear() def add_points(self, points, colors): points_item = gl.GLScatterPlotItem(pos=points, size=1.5, color=colors) self.add_item(points_item) def add_line(self, p1, p2, color, width=3): lines = np.array([[p1[0], p1[1], p1[2]], [p2[0], p2[1], p2[2]]]) lines_item = gl.GLLinePlotItem(pos=lines, mode='lines', color=color, width=width, antialias=True) self.add_item(lines_item) def plot_bbox_mesh(self, gt_boxes3d, color=(0,1,0,1)): b = gt_boxes3d for k in range(0,4): i,j=k,(k+1)%4 self.add_line([b[i,0],b[i,1],b[i,2]], [b[j,0],b[j,1],b[j,2]], color) i,j=k+4,(k+1)%4 + 4 self.add_line([b[i,0],b[i,1],b[i,2]], [b[j,0],b[j,1],b[j,2]], color) i,j=k,k+4 self.add_line([b[i,0],b[i,1],b[i,2]], [b[j,0],b[j,1],b[j,2]], color) def value_to_rgb(pc_inte): minimum, maximum = np.min(pc_inte), np.max(pc_inte) ratio = (pc_inte - minimum + 0.1) / (maximum - minimum + 0.1) r = (np.maximum((1 - ratio), 0)) b = (np.maximum((ratio - 1), 0)) g = 1 - b - r return np.stack([r, g, b]).transpose() def view_points_cloud(pc=None): app = QtGui.QApplication([]) glview = plot3d() if pc is None: pc = np.random.rand(1024, 3) pc_color = np.ones([pc.shape[0], 4]) glview.add_points(pc, pc_color) glview.view.show() return app.exec()
How can I use this code to visualize my bin files?
@Tantoyy @DavidDiosdado you can call view_points_cloud()
to display your (N,3) points array
@Tantoyy @DavidDiosdado you can call
view_points_cloud()
to display your (N,3) points array
Hi, In view_points_cloud(), how to transfer the depth bin file into the parameter "pc"?
Hi @lilingge , I have add a jupyter script for visualization in the folder ./visualization. Hope it can help you.
Hi @lilingge , I have add a jupyter script for visualization in the folder ./visualization. Hope it can help you.
Hi, author, the code is very useful! But I still have a question, how do I save the Cloud as png?
I usually take a screenshot. Or you can use PPTK library https://github.com/heremaps/pptk.
I usually take a screenshot. Or you can use PPTK library https://github.com/heremaps/pptk.
Emmm, I get trouble in showing cloud. I can't get expected interface as your show and I reopen my web browser but not make difference. What should I do?
Hi, have you installed the requirements? pip install pythreejs pyntcloud pandas numpy
The question is solved! Thank you very much! It's something else, not the requirements.
Hi, mileyan. I add
--save_float
option to runpython ./psmnet/submission.py
and generate disparity map stored in npy. In the meanwhile, modified serveral lines in./preprocessing/generate_lidar.py
to read .npy instead of .png. The result looks better. training/000000training/000002
training/000003
But i not sure whether it is correct now. I notice the estimated depth is not so good at the distance.
Hi, I don't find "save_float" in submission.py. Do you mean "save_figure" instead?
The question is solved! Thank you very much! It's something else, not the requirements.
Hello, I cannot visualize the point cloud as the interface. How to solve it? Thanks!!!
Hi, Is this how it is supposed to be now ?
![]()
![]()
But, I notice the generated pseudo point cloud seems not similar to the ones in your papers
Hello, I encountered the same problem as you did. The pseudo-point cloud I generated is also sliced. Could you please tell me how you solved this problem? Thank you.
Hi, Yan. Thanks for your great work.
I am using pytorch 1.1.0.
I follow the readme to generate disparity and point cloud. First, i download your pretrained PSMNet model. Then, i run following cmd to generate disparity.
Next, i run following cmd to generate point cloud *.bin
However, i got disparity map for
and point cloud like this:
The RGB image is:
![000002l](https://user-images.githubusercontent.com/2350713/60496778-16c60c80-9ce6-11e9-9c26-d0e4993087b3.png)
training/000002
like this:I think the generated disparity is bad. But i can't figure out the reason. Thank you