in browser 3D visualizations using ChemiScope

Dalouvid commented 1 year ago

Include a way to visualize simulations in browser to show in lectures, tutorials etc. using ChemiScope.

RudolfWeeber commented 1 year ago

Notes on potential visualization packages:

three.js

Java-script based
There are Jupyter-ready Python bindings (pythreejs), providing a Jupyter widget
A client-server approach with a Python server and pure html/js frontend without Jupyter is also possible https://www.smashingmagazine.com/2016/02/simple-augmented-reality-with-opencv-a-three-js/
A raytracer exists: https://github.com/erichlof/THREE.js-PathTracing-Renderer
Three.js is application-agnostic, so we'll have to do the particle and fluid specific stuff ourselves, or rely on third party add-ons.

Questions:

How extensible are the Python bindings in pythreejs. Can it be combined with the ray tracer, or with three.js addons e.g., for streamline plotting @pythonfz could you please expain what you've done with three.js so far?

Open3d

c++-based with Python bindings
Can stream to a web client with webrtc, but there are limitations w/r to the server (see end of introducoint in http://www.open3d.org/docs/release/tutorial/visualization/web_visualizer.html)
- the server needs a GPU (which is not present on many cheap cloud instances)
- ARM not yet supported
- Mac not yet supported

To my understandin, thesee limitatoins basically rule it out for Espresso.

PythonFZ commented 1 year ago

The ZnDraw package (https://github.com/zincware/ZnDraw) uses three.js, flask and websockets for communication. It does not rely on pythreejs and therefore has access to all three.js features. It relies on ASE (see https://github.com/espressomd/espresso/issues/4771) for the representation of particles.

A Jupyter widget, webbrowser and a pyQT / pywebview based dedicated window are available. These features are all available in the latest pre-release (see https://pypi.org/project/zndraw/#history). The stable version will be updated soon.

The visualisation in Python is handled in a list-like fashion. If you use __setitem__, append, or extend the data will be sent to three.js for visualisation. The same way, data can be retrieved.

from zndraw import ZnDraw

vis = ZnDraw()

for step in md.run():
    step: ase.Atoms
    vis.extend(step)

ZnDraw was succesfully tested on windows, linux and mac without the need for conda packages (pip install zndraw==0.2.0a12). ZnDraw already has a plotting interface through ploty. Using JSON Schema based forms, it can easily be extended by custom user inputs (easily done with pydantic, see https://docs.pydantic.dev/latest/usage/json_schema/ )

RudolfWeeber commented 1 year ago

I took a look at the source code of ZnDraw. As I understand it, we can at the very least use the js side. On teh python side, I'm not sure, we want to inherit the ase dependency for Espresso. this needs further consideration.

Do I undertand correclty, that the update is pull-based, i.e., from main.js the update_config() method is called in regular intervals and retrieves new atom positions from the server via a http request? We might have to use a push approach using web-sockets for live visualization in Espresso.

PythonFZ commented 1 year ago

I took a look at the source code of ZnDraw. As I understand it, we can at the very least use the js side.

I have updated the main branch about 1 h ago and almost everything has been rewritten.

Do I undertand correclty, that the update is pull-based, i.e., from main.js the update_config() method is called in regular intervals and retrieves new atom positions from the server via a http request? We might have to use a push approach using web-sockets for live visualization in Espresso.

The communication is bidirectional. The http requests (server sent events in this case) where pull based but have been replaced.

Configurations are sent to JavaScript using socket.emit("atoms:upload", {index: atoms_to_json(value)}). The necessary keys to send data to the JavaScript can be found in the class Atoms constructor here https://github.com/zincware/ZnDraw/blob/main/zndraw/static/pycom/Cache.js which would allow sending data (more or less) directly via websockets to JavaScript. I want to mention, that I do plan to add some restrictions to the communication, e,g. through a token or password, but this has not been added yet.

At this point, the available channels for communication are all listed in https://github.com/zincware/ZnDraw/blob/main/zndraw/app.py

On the python side, I'm not sure, we want to inherit the ase dependency for Espresso. this needs further consideration.

As mentioned in https://github.com/espressomd/espresso/issues/4771 I do think that adding ASE support would not only be beneficial for the interface to ZnDraw but also could make ESPResSo more attrative to the MLP community. If adding the dependency is the main struggle, having something like this could be interesting?

try:
   import ase
   # ESPResSo to ASE code
except ImportError as err:
   raise ImportError(
      "The ESPResSo to ASE interface requires ASE to be installed. ASE can be installed via 'pip install ase'."
   ) from err

The export to ASE should not be more than a few lines of code.

espressomd / espresso