Docker for Open3D

[takanokage]

Headless rendering on ubuntu:16.04 with Xvfb, x11vnc, lxde and bind mounting.

[takanokage]

Oh, please note that the CI can't do anything with docker at the moment. I can imagine all kinds of issues in attempting to add the docker path to CI so for the time being I recommend testing it manually. Worked for me.

[takanokage]

please hold the merge, I need to make some tweaks. for example the proxy settings are specific to my local machine and won't work anywhere else.

[syncle]

We had decided to use util for the new folder name, not utilities.

[takanokage]

ok, let me change that in a second.

[syncle]

Thanks @takanokage. Can you also make a new PR to submit a document for this? We will need instructions to use this. You can work on docs/advanced/ and make headless_rendering.rst.

[takanokage]

Yep, that was the plan from the beginning. For the purpose of this particular PR I added brief instructions to #17.

[takanokage]

I have written the headless rendering documentation page. I'm sure there are things that I missed. Please try the headless rendering demo and let me know what you would like improved. Let me know if/how I can help; I'd be happy to walk you trough if/when you get stuck.

[syncle]

Awesome. Thanks @takanokage! I am looking this PR. I will post any comments regarding this.

[syncle]

I just tested this PR. I was able to save both depth and color images. However, the saved color images are just filled with black color. Depth images looks fine. @takanokage, can you check on this?

[takanokage]

will do. could you please give me more info? how did you obtain the color images?

[syncle]

I just tried headless rendering from terminal and use the following scripts.

$ cd <Open3D path>/utilities/docker/ubuntu-xvfb/tools
$ ./build.sh
$ ./attach.sh
$ ./headless_sample.sh

[takanokage]

I'm doing the same thing but it looks like I'm only getting depth images in /Open3D/build/lib/TestData/depth. Did you do anything else?

[syncle]

Nope. I did not tweak anything.

I'm only getting depth images in /Open3D/build/lib/TestData/depth

I think you are also having same trouble. My case saves color images in /Open3D/build/lib/TestData/color, but they are just black images.

[takanokage]

ok, I confirm the images in /Open3D/build/lib/TestData/image are black. There seems to be a problem with visualizer.capture_screen_float_buffer(...). What's your opinion? I'm still ramping up on the Open3D code so any pointers would help.

[takanokage]

Another observation: I'm getting similar behavior (black images) when operating Open3D over VNC.

[qianyizh]

visualizer.capture_screen_float_buffer() is a wrapper of this function:

https://github.com/IntelVCL/Open3D/blob/master/src/Visualization/Visualizer/VisualizerRender.cpp#L147-L174

It is weird because it should be no difference to visualizer.capture_depth_float_buffer(): https://github.com/IntelVCL/Open3D/blob/39e432b5cd2179725cc5403b5d53255f3eb41b71/src/Visualization/Visualizer/VisualizerRender.cpp#L223-L285

Maybe try to use capture_screen_image and capture_depth_image first to see if it is a problem with headless rendering / glReadPixels. If not, then it is probably a bug. Need to go into the C++ functions and debug.

[takanokage]

capture_depth_image works. capture_screen_image doesn't work (saves black images).

[syncle]

I believe we need to revisit GLFW3.3 + OSMesa Option. With GLFW3.2, I am having

RandR gamma ramp support seems broken

Errors, and that is probably the reason. As @takanokage said, this may occur segfault errors though.

[qianyizh]

I thought we have discussed this in #17 and have decided to use only glfw3.3 dev branch for headless rendering?

[syncle]

True. I prefer GLFW3.3 dev. Dan tried this option before and he decide to keep using GLFW3.2 due to technical reason. This PR still based on GLFW 3.2. However, I believe this issue should be addressed with GLFW3.3 dev.

[takanokage]

well, we've discussed several topics related to #17.

a. Jaesik worked with glfw3.3 in order to get to headless rendering. I didn't try that path and don't know if it works. b. I worked with Docker in order to get to headless rendering. glfw3.3 crashes in the Docker setup but glfw3.2 doesn't.

[takanokage]

I'd also like to point out that rendering works just fine as confirmed by VNC into the Docker container.

[takanokage]

I propose we break this down into two parts:

1. the current PR
2. another PR for the limitations/issues of the current headless solution.

Advantages:

• people get access to some headless functionality, they get to try it out and maybe even contribute to it.
• the current PR doesn't get delayed for any additional issue we find with the current solution; it's going to take a while for me to identify what the problem is with saving rendered images.

If we do this we need to list the current known limitations in a "release" doc.

[syncle]

I am revisiting option a) GLFW 3.3 dev + OSMesa. The segmentation fault comes from glew. Open3D's glew is outdated and does not support OSMesa. Let me keep working on this issue.

[takanokage]

that sounds great, good luck!

[syncle]

Good news! glew2.1 + OSMesa18.0 + glfw3.3 dev works for headless rendering. I was able to generate rendering images from terminal via clean Ubuntu 16.04. I don't have any black screen images. I don't need to make a fake display. It just runs out-of-box.

I will need to polish how I did: dependencies, modifying cmake tool chains, adding switch "headless rendering" for selective compilation. This switch will feed one of graphical library (OpenGL or OSMesa) for glew and glfw.

Edit: This method is addressed in #273

[syncle]

@takanokage, Can you remove rendering from termal part in this PR? I think this PR need to be polished as a generic docker image for Open3D.

[takanokage]

how about we close this PR and start a new one once all the polishing is done?

[syncle]

Sure. It may look not that admirable if a PR is opened but not merged for long time.

[takanokage]

closing until #273 is over.