Testing web browser backend

ejeschke commented 9 years ago

Ginga has a new backend, targeting canvases in HTML5 compliant browsers. I am looking for early feedback on this capability. You will need at least commit 97ec3d5e25a4c77498138d3db46cf74b3a20772e or later.

The easiest test right now is example2_pg.py Run it like this:

    $ python .../path/to/ginga/examples/pg/example2_pg.py --loglevel=20 --stderr path/to/some/fitsfile.fits

The very last line of the log output should tell you a URL to visit with an HTML5/websockets compliant browser. Visit this page and click on the link for the Ginga window. You should get something that looks like this: ginga_web_screenshot

But of course with your FITS file.

Helpful additional packages

For full functionality with graphical overlays you will need one of {aggdraw|PIL/pillow||opencv (cv2)} modules installed on the server side. PIL is a pretty common library included in many distributions. aggdraw provides the most complete experience similar to the native widgets (qt/gtk/tk)--you can get it here; currently it only supports python 2. opencv provides a huge boost to rotation performance, no matter what what widget set you are using. If you have it installed, add the --opencv flag to the command invocation.

Non-local access

The example program also supports the --host and --port options for listening on alternate interfaces. Start the example with:

--host=

i.e. empty host to have the example listen on all interfaces. You can then connect from any other system that can connect to this host.

Sharing

Windows are shareable. This has interesting applications for collaboration. To test it, simply open another browser window to the same address, possibly from a different host.

Trackpad

In the example, there is a checkbox for "I'm using a trackpad". Check it if you are, you will have a more pleasant experience zooming the window.

ejeschke commented 9 years ago

@pllim, @stscieisenhamer, @nabobalis, @Cadair, @rupak0577 Would be very interested in your feedback testing this!

ejeschke commented 9 years ago

The quick reference may be handy for those unfamiliar with the keyboard and mouse bindings available in the basic widget.

ejeschke commented 9 years ago

Sorry, you may need commit f45ed135db4572907eef0e4c47fb734a2de7a312 to follow the instructions as I wrote above. This is now in master.

Cadair commented 9 years ago

Oh this is cool! I will look later today.

Cadair commented 9 years ago

@dpshelio think of solar monitor.

nabobalis commented 9 years ago

@ejeschke This is seriously awesome. I assume there is no access to other plugins right now?

rupak0577 commented 9 years ago

@ejeschke Nice! There is a slight problem though - the buttons are missing.

Would love to help in any way possible.

EDIT : I found the buttons - the scrollbar is not present so I had to middle-click drag to see them.

pllim commented 9 years ago

:+1: Started up fine. Looks very promising!

ejeschke commented 9 years ago

You can also use it in the current form with ipython notebooks: see here

ejeschke commented 9 years ago

@nabobalis, no plugins yet, but the plan is to support a version of the reference viewer running in the browser, if possible.

stscieisenhamer commented 9 years ago

:+1: works good in chrome! nice!

pllim commented 8 years ago

I believe @eteq gave a pretty good demo about this feature at the STScI sprint. :smiley:

If you are interested, proof of concept is available at https://github.com/spacetelescope/stginga/blob/master/stginga/examples/ginga_nbinteract.ipynb

eteq commented 8 years ago

One thing to clarify on that (for anyone else who might be reading this): I'm particularly interested in using this inside Jupyter/IPython notebooks. In that use case, it's an advantage to have the viewer itself be fairly limited in terms of GUI features, because you can access it all in the notebook. But then it's more important to have easy-to-use programmatic interfaces, and that's what the stuff I've been working on in stginga is about.

eteq commented 8 years ago

Oh, and that leads to some feedback for @ejeschke - hopefully this backend can stay fairly segmented in the same way that the reference viewer is: i.e. the basic viewer itself shouldn't be too tightly dependent on the context/extra GUI interfaces that add bells and whistles like the drawing options you're showing in the screenshot above. That's nice in some cases (even in a notebook context), but it shouldn't be necessary. Along the same lines, I think it's useful to try to keep the GUI<->ginga interaction well-documented, because that will make it possible for users (not devs) to very quickly design special-purpose notebook-based "GUIs" for a particular workflow.

My reading of the code is that this is indeed the intent, but thought I would share this just to have in mind.

ejeschke commented 8 years ago

@eteq, totally in agreement here. I call it a toolkit for building image viewers and that's the goal, really. The people that have generally expressed interest are the ones building things. But you are right, in the notebook context it could really open things up for experimentation for a more general, less developer-oriented type of individual.

sosey commented 8 years ago

Tkinter libs conflict with Anaconda Python2/3 and OSX system install

@ejeschke I'm sticking this here incase you have anyone run into the same issue. This happens in python 2/3 and causes major badness in the form of a kernel abort. I'm having a hell of time getting around it. The problem is applications building with the system install of TK instead of the local (for example anaconda) library, detailed for pillow here:

https://github.com/python-pillow/Pillow/pull/1883

Anaconda installs pillow and tk into the root, and tk is a dependency for everything that uses x11 as far as I know.

I get this importing imexam, which is the symptom:

objc[42712]: Class TKApplication is implemented in both     /System/Library/Frameworks/Tk.framework/Versions/8.5/Tk and /Users/sosey/use_anaconda3/envs/astroconda/lib/libtk8.5.dylib. One of the two will be used. Which one is undefined.
objc[42712]: Class TKMenu is implemented in both /System/Library/Frameworks/Tk.framework/Versions/8.5/Tk and /Users/sosey/use_anaconda3/envs/astroconda/lib/libtk8.5.dylib. One of the two will be used. Which one is undefined.
objc[42712]: Class TKContentView is implemented in both /System/Library/Frameworks/Tk.framework/Versions/8.5/Tk and /Users/sosey/use_anaconda3/envs/astroconda/lib/libtk8.5.dylib. One of the two will be used. Which one is undefined.
objc[42712]: Class TKWindow is implemented in both /System/Library/Frameworks/Tk.framework/Versions/8.5/Tk and /Users/sosey/use_anaconda3/envs/astroconda/lib/libtk8.5.dylib. One of the two will be used. Which one is undefined.

which results in this when the gods hate you:

Do you really want to exit ([y]/n)? y
alloc: invalid block: 0x10a1e2bd0: 10 0

/Users/sosey/anaconda3/envs/notk/bin/python.app: line 3: 53664 Abort trap: 6           /Users/sosey/anaconda3/envs/notk/python.app/Contents/MacOS/python "$@"

I'm trying some of the workarounds I've seen mentioned, but hopefully they'll be a better fix in the near future, or if you hear of one let me know :) I only just ran into this after I had to wipe my Anaconda installations from my laptop and reinstall everything.

ejeschke commented 8 years ago

@sosey, I don't see this problem on my mac (os x 10.11.5) with anaconda installed. Here's what I have:

MacBook-Pro-2:qplan eric$ conda list | grep tk
nltk                      3.1                      py27_0  
tk                        8.5.18                        0  
MacBook-Pro-2:qplan eric$ conda list | grep pillow
pillow                    3.2.0                    py27_0

Can you give me some more information about your environment?

ejeschke commented 8 years ago

@sosey, are your running from a terminal or jupyter notebook? Could you try setting the env variable MPLBACKEND before however you are running? e.g.

export MPLBACKEND="Qt4agg"

Just to make sure any matplotlib imports don't try to use Tk. Let me know your result. Tnx.

sosey commented 8 years ago

@ejeschke sorry for the very long delay! I've been swamped.

Just to be clear, it's not a ginga or imexam issue really, it's an installation confusing issue I believe.

Yah, I can avoid the issue using the Qt4Agg background, as long as I don't install pillow. So it's okay for imexam with ds9. I was trying to find a workaround for users so that they could use tkagg, which many probably have as a default.

However, I still have issues with ginga, I can't use tkagg for the reasons above, but even with qt4agg:

In a python2 env with these installed:

# packages in environment at /Users/sosey/anaconda3/envs/python2:
#
appnope                   0.1.0                    py27_0  
astropy                   1.2.1               np111py27_0  
backports                 1.0                      py27_0  
backports_abc             0.4                      py27_0  
cycler                    0.10.0                   py27_0  
cython                    0.24                     py27_0  
freetype                  2.5.5                         1  
get_terminal_size         1.0.0                    py27_0  
ginga                     2.5.20160627100500           <pip>
decorator                 4.0.9                    py27_0    http://ssb.stsci.edu/astroconda
imexam                    0.6.dev220                <pip>
ipython                   5.0.0                    py27_0  
ipython_genutils          0.1.0                    py27_0  
libpng                    1.6.22                        0  
matplotlib                1.5.1               np111py27_0  
mkl                       11.3.3                        0  
numpy                     1.11.1                   py27_0  
openssl                   1.0.2h                        1  
path.py                   8.2.1                    py27_0  
pathlib2                  2.1.0                    py27_0  
pexpect                   4.0.1                    py27_0  
pickleshare               0.7.2                    py27_0  
pip                       8.1.2                    py27_0  
prompt_toolkit            1.0.3                    py27_0  
ptyprocess                0.5.1                    py27_0  
pygments                  2.1.3                    py27_0  
pyparsing                 2.1.4                    py27_0  
pyqt                      4.11.4                   py27_3  
python                    2.7.12                        1  
python-dateutil           2.5.3                    py27_0  
python.app                1.2                      py27_4  
pytz                      2016.4                   py27_0  
qt                        4.8.7                         3  
readline                  6.2                           2  
scipy                     0.17.1              np111py27_1  
setuptools                23.0.0                   py27_0  
simplegeneric             0.8.1                    py27_1  
singledispatch            3.4.0.3                  py27_0  
sip                       4.16.9                   py27_0  
six                       1.10.0                   py27_0  
sqlite                    3.13.0                        0  
ssl_match_hostname        3.4.0.2                  py27_1  
tk                        8.5.18                        0  
tornado                   4.3                      py27_1  
traitlets                 4.2.1                    py27_0  
wcwidth                   0.1.7                    py27_0  
wheel                     0.29.0                   py27_0  
zlib                      1.2.8                         3

After specifying the Qt4Agg backend first, when I try to start a ginga viewer I get this error:

/Users/sosey/anaconda3/envs/python2/lib/python2.7/site-packages/ginga-2.5.20160627100500-py2.7.egg/ginga/util/io_rgb.py in <module>()
     37 if not have_pilutil:
     38     try:
---> 39         from ginga.qtw.QtHelp import QImage, QColor, QtCore
     40         have_qtimage = True
     41     except ImportError as e:

/Users/sosey/anaconda3/envs/python2/lib/python2.7/site-packages/ginga-2.5.20160627100500-py2.7.egg/ginga/qtw/QtHelp.py in <module>()
     49         import sip
     50         for cl in ('QString', 'QVariant'):
---> 51             sip.setapi(cl, 2)
     52 
     53         from PyQt4 import QtCore, QtGui

ValueError: API 'QString' has already been set to version 1

In python3.5.2, I can start up a viewer, but I can't see the image without installing pillow. Once pillow is installed in the environment, I crash python and my terminal when I try and make a plot, even if I specify the qt4agg backend ahead of time. Here's my python 3.5 env:

# packages in environment at /Users/sosey/anaconda3/envs/python35:
#
appnope                   0.1.0                    py35_0  
astropy                   1.2.1               np111py35_0  
cycler                    0.10.0                   py35_0  
cython                    0.24                     py35_0  
freetype                  2.5.5                         1  
ginga                     2.5.20160627100500           <pip>
decorator                 4.0.9                    py35_0    http://ssb.stsci.edu/astroconda
imexam                    0.6.dev220                <pip>
ipython                   5.0.0                    py35_0  
ipython_genutils          0.1.0                    py35_0  
libpng                    1.6.22                        0  
matplotlib                1.5.1               np111py35_0  
mkl                       11.3.3                        0  
numpy                     1.11.1                   py35_0  
openssl                   1.0.2h                        1  
path.py                   8.2.1                    py35_0  
pexpect                   4.0.1                    py35_0  
pickleshare               0.7.2                    py35_0  
pip                       8.1.2                    py35_0  
prompt_toolkit            1.0.3                    py35_0  
ptyprocess                0.5.1                    py35_0  
pygments                  2.1.3                    py35_0  
pyparsing                 2.1.4                    py35_0  
pyqt                      4.11.4                   py35_3  
python                    3.5.2                         0  
python-dateutil           2.5.3                    py35_0  
python.app                1.2                      py35_4  
pytz                      2016.4                   py35_0  
qt                        4.8.7                         3  
readline                  6.2                           2  
scipy                     0.17.1              np111py35_1  
setuptools                23.0.0                   py35_0  
simplegeneric             0.8.1                    py35_1  
sip                       4.16.9                   py35_0  
six                       1.10.0                   py35_0  
sqlite                    3.13.0                        0  
tk                        8.5.18                        0  
tornado                   4.3                      py35_1  
traitlets                 4.2.1                    py35_0  
wcwidth                   0.1.7                    py35_0  
wheel                     0.29.0                   py35_0  
xz                        5.2.2                         0  
zlib                      1.2.8                         3

ejeschke commented 8 years ago

@sosey, sorry for late reply also--am on vacation but returning soon. For py2.7 case, I have run into this before in programs that use both matplotlib and ginga packages together with Qt--ginga wants to use Qt API v2, but if you import matplotlib first it will try to go to v1 API by default. The solution I have found is to either import ginga before matplotlib, or to set the env variable QT_API=pyqt before importing matplotlib for the first time. I think if you do the latter, it won't force matplotlib to use Qt, but if it does choose Qt it will choose the v2 API. Could you see whether setting this variable makes a difference in the 2.7 case?

We'll have to dig a little deeper to see why you are getting the problem with python 3, but perhaps the solution suggested in #348 would solve this problem for both python 2 and 3.

sosey commented 8 years ago

Thanks @ejeschke, I'll give those suggestions a go. Also - don't work on your vacation! :)

ejeschke commented 8 years ago

@sosey, can you check whether the problem with the Tk/PIL version is still present? I've merged #362, which fixed a problem with that particular combination.

sosey commented 8 years ago

@ejeschke - here's the current status. setting QT_API does help for running python35 in the notebook, but I get kernel crashes from an ipython terminal. Here are the different modes I've tried and the results. I think there's still a threading issue somewhere since I get differing results with the notebook vs an ipython terminal.

with python35, ipython terminal, tkagg backend:

I get a kernel crash, and on import I get messages about multiple tk backends loaded. HOWEVER, if I uninstall the conda built matplotlib and pip install matplotlib then the multiple tk backend warnings go away when I import. Both ways I have to reload the HTML window to get the image to display. With the matplotlib from pip loaded the kernel crash goes away and the plots show up fine. BUT when I go to quit out of the ipython terminal it hangs until I ^C:

Do you really want to exit ([y]/n)? y
^CException ignored in: <module 'threading' from '/Users/sosey/miniconda3/envs/python35/lib/python3.5/threading.py'>
Traceback (most recent call last):
  File "/Users/sosey/miniconda3/envs/python35/lib/python3.5/threading.py", line 1288, in _shutdown
    t.join()
  File "/Users/sosey/miniconda3/envs/python35/lib/python3.5/threading.py", line 1054, in join
    self._wait_for_tstate_lock()
  File "/Users/sosey/miniconda3/envs/python35/lib/python3.5/threading.py", line 1070, in _wait_for_tstate_lock
    elif lock.acquire(block, timeout):
KeyboardInterrupt
Exception ignored in: <bound method Image.__del__ of <tkinter.PhotoImage object at 0x1142abfd0>>
Traceback (most recent call last):
  File "/Users/sosey/miniconda3/envs/python35/lib/python3.5/tkinter/__init__.py", line 3356, in __del__
    self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop

with python35, jupyter notebook, tkagg backend: image loads immediately, plots show up in the notebook when I specify the nbagg backend at start too. the tkagg backend opens and closes plots without issue as well. Shutting down the notebook does not require a ^c

with python35, qt4agg, ipython terminal when I try and display an image in the html window, the viewer goes white and I get this error:

In [5]: ERROR:tornado.application:Uncaught exception GET /app/socket?wid=Imexam%20Display (::1)
HTTPServerRequest(protocol='http', host='localhost:1245', method='GET', uri='/app/socket?wid=Imexam%20Display', version='HTTP/1.1', remote_ip='::1', headers={'Sec-Websocket-Key'
Traceback (most recent call last):
  File "/Users/sosey/miniconda3/envs/python35/lib/python3.5/site-packages/tornado/web.py", line 1425, in _stack_context_handle_exception
    raise_exc_info((type, value, traceback))
  File "<string>", line 3, in raise_exc_info
  File "/Users/sosey/miniconda3/envs/python35/lib/python3.5/site-packages/tornado/web.py", line 1627, in wrapper
    result = method(self, *args, **kwargs)
  File "/Users/sosey/miniconda3/envs/python35/lib/python3.5/site-packages/tornado/websocket.py", line 180, in get
    self.stream.set_close_callback(self.on_connection_close)
  File "/Users/sosey/miniconda3/envs/python35/lib/python3.5/site-packages/tornado/iostream.py", line 412, in set_close_callback
    self._maybe_add_error_listener()
  File "/Users/sosey/miniconda3/envs/python35/lib/python3.5/site-packages/tornado/iostream.py", line 901, in _maybe_add_error_listener
    self._add_io_state(ioloop.IOLoop.READ)
  File "/Users/sosey/miniconda3/envs/python35/lib/python3.5/site-packages/tornado/iostream.py", line 931, in _add_io_state
    self.fileno(), self._handle_events, self._state)
  File "/Users/sosey/miniconda3/envs/python35/lib/python3.5/site-packages/tornado/ioloop.py", line 725, in add_handler
    self._impl.register(fd, events | self.ERROR)
  File "/Users/sosey/miniconda3/envs/python35/lib/python3.5/site-packages/tornado/platform/kqueue.py", line 40, in register
    raise IOError("fd %s already registered" % fd)
OSError: fd 23 already registered

I then have to ^c out of the session

python35, qt4agg, jupyter notebook: no problem starting up the notebook and viewing plots, no errors, works with nbagg backend as well to put plots into the notebook.

austinbeauch commented 7 years ago

Hi @ejeschke,

I've been working with Ginga's web backend as in example2_pg.py and noticed there might be a slight hiccup when creating the viewer widget as on line 75 of example2_pg.py:

w = Viewers.GingaViewerWidget(viewer=fi)

There's no issue there, but trying to create a different sized window using a command such as

w = Viewers.GingaViewerWidget(viewer=fi, width=1000, hiehgt=1000)

didn't work and I noticed that lines 15/16 of Viewers.py could be altered in the slightest way.

def __init__(self, viewer=None, width=600, height=600):
   super(GingaViewerWidget, self).__init__(width=600, height=600)

could be changed to

def __init__(self, viewer=None, width=600, height=600):
   super(GingaViewerWidget, self).__init__(width=width, height=height)

so that the default values could be changed. It's the smallest change and I doubt anyone will ever make use of it (I'm not even sure if we need any different sizes other than 600x600), but I could make a quick pull request with the change if you wanted!

ejeschke commented 7 years ago

@austinbeauch, I'd be happy to receive your PR!

ejeschke / ginga