No Image when using interface.py on Google Colab

[DougShuffield]

Using https://github.com/wpmed92/3d-ken-burns-colab as the base, I added the following to the end of the notebook to attempt to use interface.py so I can do my own camera paths:

#Get an internet accessible address to the local server from interface.py. Changed server port in interface.py to 8050 from google.colab.output import eval_js print(eval_js("google.colab.kernel.proxyPort(8050)")) # Will be something like: https://z4spb7cvssd-496ff2e9c6d22116-8050-colab.googleusercontent.com/

#Run the interface !python interface.py

Everything seems to be setup right and allows me to load an image from my PC, modify the zooms, etc., However, the finished 3D image never appears. Here is the log from interface.py:

127.0.0.1 - - [2020-05-24 18:49:04] "GET / HTTP/1.1" 200 9154 0.001346 127.0.0.1 - - [2020-05-24 18:49:04] "GET /favicon.ico HTTP/1.1" 404 356 0.001295 127.0.0.1 - - [2020-05-24 18:49:15] "POST /update_mode HTTP/1.1" 400 318 0.000623 127.0.0.1 - - [2020-05-24 18:49:15] "POST /load_image HTTP/1.1" 400 318 0.001246 127.0.0.1 - - [2020-05-24 18:49:15] "POST /update_mode HTTP/1.1" 400 318 0.000806 127.0.0.1 - - [2020-05-24 18:49:15] "POST /update_mode HTTP/1.1" 400 318 0.000689 127.0.0.1 - - [2020-05-24 18:49:15] "POST /update_mode HTTP/1.1" 400 318 0.000638 127.0.0.1 - - [2020-05-24 18:49:15] "POST /update_from HTTP/1.1" 400 318 0.000629 127.0.0.1 - - [2020-05-24 18:49:15] "POST /update_mode HTTP/1.1" 400 318 0.000582 127.0.0.1 - - [2020-05-24 18:49:16] "POST /update_mode HTTP/1.1" 400 318 0.000783 127.0.0.1 - - [2020-05-24 18:49:16] "POST /update_from HTTP/1.1" 400 318 0.000632 127.0.0.1 - - [2020-05-24 18:49:16] "POST /update_mode HTTP/1.1" 400 318 0.000649 127.0.0.1 - - [2020-05-24 18:49:16] "POST /update_from HTTP/1.1" 400 318 0.000658 127.0.0.1 - - [2020-05-24 18:49:16] "POST /update_from HTTP/1.1" 400 318 0.000727 127.0.0.1 - - [2020-05-24 18:49:16] "POST /update_to HTTP/1.1" 400 318 0.001202 127.0.0.1 - - [2020-05-24 18:49:17] "POST /update_mode HTTP/1.1" 400 318 0.000553 127.0.0.1 - - [2020-05-24 18:49:17] "POST /update_to HTTP/1.1" 400 318 0.000643 127.0.0.1 - - [2020-05-24 18:49:17] "POST /update_to HTTP/1.1" 400 318 0.000613 127.0.0.1 - - [2020-05-24 18:49:17] "POST /update_to HTTP/1.1" 400 318 0.000714 127.0.0.1 - - [2020-05-24 18:49:17] "POST /update_mode HTTP/1.1" 400 318 0.000655 127.0.0.1 - - [2020-05-24 18:50:59] "GET /get_live HTTP/1.1" 200 36695563 115.058576

And here is what the screen shows from the browser tab showing the interface:

Any help to get this to work is much appreciated.

Doug

[sniklaus]

Thank you for the detailed documentation! I was able to replicate your findings and my best guess is that Colab's proxy is unable to handle the /get_live request since this request is actually a stream and does not terminate. Since I am not super familiar with Colab, I am afraid that I do not know how to persuade Colab's proxy to forward such request. If anyone has any suggestions then Doug and I would be grateful for any hints.

[Roemer]

It actually seems to work you just have to wait quite long. One issue there is, is that the requests to update_from/to are unthrottled. I added a jquery throttler (https://github.com/cowboy/jquery-throttle-debounce) and that makes it more responsive. I also adjusted the interface to take a port as a parameter. If wanted, I can create a PR with those changes.

[sniklaus]

Thank you for looking into this and sharing your findings! It would be great if you could fork this project and commit your changes there. I would be happy to point the readme to your fork such that people interested in this setup can make use of your modifications.

[Roemer]

So I did some more research. The live stream that should be there does not work. I think this is some kind of colab limitation. What I tried now is to add a toggle button to switch between the stream preview and a still image which is refreshed on every update call. It works actually fairly well and allows the use of colab to view a preview. Surely not as smooth as with the stream but definitely better than nothing. I will open a PR for it, you can close it if you don't want to merge any changes.

[edenraf]

@Roemer would you be able to post a little more code so I could replicate the interface on my colab? I tried using the code you posted at the top but get the following error. Perhaps I need to modify the interface.py to expect the google colabs link instead of localhost? Any help greatly appreciated!

Traceback (most recent call last): File "interface.py", line 213, in <module> gevent.pywsgi.WSGIServer(listener=('0.0.0.0', 8080), application=objFlask).serve_forever() File "/usr/local/lib/python3.7/dist-packages/gevent/baseserver.py", line 398, in serve_forever self.start() File "/usr/local/lib/python3.7/dist-packages/gevent/baseserver.py", line 336, in start self.init_socket() File "/usr/local/lib/python3.7/dist-packages/gevent/pywsgi.py", line 1545, in init_socket StreamServer.init_socket(self) File "/usr/local/lib/python3.7/dist-packages/gevent/server.py", line 180, in init_socket self.socket = self.get_listener(self.address, self.backlog, self.family) File "/usr/local/lib/python3.7/dist-packages/gevent/server.py", line 192, in get_listener return _tcp_listener(address, backlog=backlog, reuse_addr=cls.reuse_addr, family=family) File "/usr/local/lib/python3.7/dist-packages/gevent/server.py", line 288, in _tcp_listener sock.bind(address) File "/usr/local/lib/python3.7/dist-packages/gevent/_socketcommon.py", line 563, in bind return self._sock.bind(address) OSError: [Errno 98] Address already in use: ('0.0.0.0', 8080)

[edenraf]

For anyone else looking into this, I have found a fix. The error message comes from a websocket that is already open. I assume google colabs is using the 8080 port for something else. So, I went into interface.py to line 212 and changed the port to 1234. Then change the port call print(eval_js("google.colab.kernel.proxyPort(1234)")) Done

[sniklaus]

Huge thanks for sharing this information! I am hence closing this issue for now. :+1: