Add IPython Widgets

[ax3l]

Add IPython widget for controlling the sim (ISAAC feedback channel).

Maybe lend from ipyvolume's overlay for mouse interactions (or use it somehow directly?), otherwise use the same control as used in gamepad/joystick widgets.

Scale Image size to visible area / widget size and make it resizable.

[maartenbreddels]

Some ideas that come to mind:

• Use (optional) webrtc for transportation of data, which will take care of firewall penetration.
  • Have a widget that is a (subclass) of ipywebrtc.MediaStream that receives the data over webrtc
  • A widget that uses you own protocol (I saw base64 encoded png images I think)
• Have a canvas based widget that takes a ipywebrtc.MediaStream (and draws it on the canvas). Mouse events are send as events, possible over your own protocol layer (but also webrtc should support that).
• Have some hybrid approach, where a high res image is send, and a low res volumetric cube. Every time the camera is rotated, ipyvolume renders the low res version, once the camera doesn't move for X msec, request the high res from isaac.

[ax3l]

Update: if we want to couple directly on the C++ side as a widget, which would spare us a lot of port-forwarding for the image and control streams: https://github.com/QuantStack/xvolume