Thousands of molecules with surface mesh

[nantille]

Hello there,

For a current project, I need to be able to import thousands of molecules and control them (Empties are welcome for that matter) according to a simulation a scientist provided me with.

So I need surface meshing of molecules and I can't find how to do that. So far the only thing I'm able to do is to load a molecule in an atomic representation. Does surface representation work for you ? If it does, could you mention how you do it ?

This now leads me to the second part: performance. Seeing how Blender is slowed down by a single more or less complex molecule, I fear that thousands of molecules are out of question here.

Is BioBlender intended for small-scale simulation only ? Was is the typical amount of molecules you handle when working with BioBlender ?

[zeffii]

@nantille Hi!

For surface representation currently BioBlender uses PyMOL. If you don't already have it installed I've made a step by step guide that covers (all) the things I needed to do to install it. If PyMOL is on your operating system's search path, BioBlender should be able to make calls to it.

Reading the pdb is fast, the creation of the Atoms and configuration of the motion mechanics is the bottle-neck - this gets progressively worse for larger files. This is a big concern and no immediate solution exists, but a proposal is to delegate the molecular dynamics to an external app too and import all atoms as point clouds could be done. Using a point cloud cache-per-frame would allow fast instancing and lower memory usage.

Maybe @MonZop would like to express her intentions for BioBlender, known limitations vs perceived limitations vs big-picture wishes.

[MonZop]

Hi, we realize that the issue of importing large molecules is one problem of BioBledner at this time. Before addressing it with an alternative solution (point cloud), we are trying to get to a point where all features of BioBlender work (albeit slowly) If you have thousands of molecules, and if you only want to visualize them all, my suggestion would be that you import it once, make the surface mesh, and then replicate the mesh. This is the way we used to build, for example, the microtubule, by serial duplication of the alpha/beta tubulin.

[nantille]

I have done some research last week about all the libraries/frameworks out there that handle pdb molecules. Among the very interesting ones that can be automated, there is PyMol, Chimera, Ugene and VMD. Pymol is indeed interesting – and I have it working on Mac. I noticed however that the generated meshes are made of unconnected polygons, meaning they are all not stitched together but this can be easily dealt with either Meshlab or Blender itself.

Using proxies is usually a good way to reduce the complexity of a scene but it depends on the amount of molecules that can be proxied (replicated). I am currently working on setting a level of detail system that uses Blender's functions to reduce tesselation based on the distance to the camera. Our simulations will grow exponentially in amount of molecules so addressing a few thousand molecules at once is a first step.

I would argue that reading a pdb is not so fast as this isn't a binary format. But you're right that compared to the meshing part in Blender, reading a pdb file is fast.

My current approach is to use pymol for the meshing and then handling the rest in Blender. BioBlender is currently too slow for me to use it in such a simulation unfortunately. Of course I lose the submolecular motion mechanics of BioBlender on the way...

[pmadhikar]

I would like to add a few comments here. I run molecular dynamics simulations all the time and visualize them regularly. And I normally use VMD for that. AFAIK VMD is a standalone program and does not come with libraries to use with third party apps. @nantille

For a current project, I need to be able to import thousands of molecules and control them (Empties are welcome for that matter) according to a simulation a scientist provided me with.

I would recommend that you ask the scientist what exactly they want. BioBlender should only be used for visualizations. There are programs out there that are designed to simulate molecules and model their interactions accurately. This is a very vibrant field of science with many approaches, and I can tell you that Blender is not the correct software for this.

If you can find out what exactly the scientists want to visualize, you can basically ignore reading/meshing a vast majority of atoms, see below for details.

@zeffii

delegate the molecular dynamics to an external app too and import all atoms as point clouds could be done

This is just not possible with any worth while simulation runs. These simulations are run on supercomputers with 10-1000 cpus, can have millions of atoms, over many months, and they can generate TBs of data. In the MD world, the simulation and visualization are done separately. Simulations generate a file with the positions of each atom at each time step. This file is processed and only a small fraction of the atoms are chosen for visualization. E.g. if the simulation was done in water, it is common to ignore all the water molecules when generating a visualization. Only then is visualization even possible. So BioBlender should only support visualizing the output after a simulation has run, not in real time.

[zeffii]

So BioBlender should only support visualizing the output after a simulation has run, not in real time.

importing trajectories and displaying many millions of atoms using Dupliverts is not beyond Blender's capabilities (using dupliverts). I'm eager to code such routines. If you have a few smaller files for me to experiment with ...

[pmadhikar]

@zeffii I can certainly give you such a file. Is there an easy way to share it here?

[zeffii]

would it fit into a gist?

[pmadhikar]

https://gist.github.com/pmadhikar/6204d58e60766771cda2#file-gistfile1-txt

There you go, it contains over 32000 atoms. The format is below. The names in the first columns are not necessarily standard, but they are unique for each atom. This system contains some water (a water model with four points) and some chloride ions. It used to have a protein in it too, but I deleted it as that can get quite complicated with looking up the structure of each amino acid.

Comment line of first frame (normally ignored during analysis) line with number of atoms in the frame atom id atom type num x_coord y_coord z_coord .... .... comment line of second frame

[zeffii]

cool, i'll have a pop at this.

[pmadhikar]

Here is what it looks like in VMD. The low density part in the middle is where the protein was.

[zeffii]

the gistfile is only one frame?

[zeffii]

oh, i just read

First frame of an MD simulation

[zeffii]

this uses Blender openGL python wrapper as a preview, tho it lacks the finesse of fog for distance (I didn't add it to the viewer yet)

https://gist.github.com/pmadhikar/6204d58e60766771cda2#comment-1407717 minimal code

[pmadhikar]

That looks really good!

[zeffii]

I just noticed how to fix a bug

[zeffii]

(tho I must stress this uses Sverchok, not BioBlender)

While Sverchok is developed with architecture and design in mind, ultimately it's just displaying geometry and uses the Blender PyNode api

image

I don't wish to derail the topic by going into any deeper discussion about Sverchok here, but a lot has happened since development stopped in BioBlender and maybe a PyNodes addition could be useful. A discussion for a later stage perhaps.

[MonZop]

I have just lookd at Sverchok, and (as expected) I am not sure I have clear what it does. However, if it can be useful for introducing in Blender (and Bioblender) trajectories of molecular dynamics, it can be a good idea to consider it in the future. The main BioBledner topic is visualization, that's why we are insisting on the surface, MLP and EP. If we can get these working in BB2.0, then we can discuss about further developments. The issue of viewing molecules in activity is very important because this is what they do: move all the time; and to understand how they work, to see them moving is very helpful. However, looking at MD simulation is difficult due to the overwhelming amount of vibration, that obscures the more 'meaningful' movements. This is one reason why we used the Blender GE to calculate some movements. The calculated motions can be physically sound, as described in our paper here