search

Eigenstate / vmd-python

Installable VMD as a python module
Other
129 stars 24 forks source link

Running into build problems trying to build for an IBM Power9 system #22

Open austinpagan opened 5 years ago

austinpagan commented 5 years ago

Before I dig too deep into this, I just wanted to know whether vmd-python was ever intended to run on Power systems...? If so, I will append the particular problem I'm seeing.

Eigenstate commented 5 years ago

Oh boy... yes regular VMD will run on anything, including Power8. When making vmd-python I removed a lot of support for architectures I didn't feel comfortable supporting. Post your error and we can totally get it working!

austinpagan commented 5 years ago

First problem I ran into is that in /usr/include/gnu, the stubs.h file was trying to include stubs-32.h, but that file was not present. I installed everything I could think of, and couldn't get that fixed. So I just hacked stubs.h to point to the file which WAS present (stubs-64-v2.h).

Then, when I tried to do "python setup.py build" again, I got this:

[root@234bc51125a5 vmd-python]# python setup.py build running build running build_py package init file 'vmd/init.py' not found (or not a regular file) package init file 'vmd/init.py' not found (or not a regular file) Compiling VMD Finding libraries... LIB: libtcl8.5.so -> /lib64 INC: tcl.h -> /usr/include WARNING: Could not find library file 'libsqlite3.so' in standard library directories. Defaulting to: '/usr/lib/libsqlite3.so' LIB: libsqlite3.so -> /usr/lib

WARNING: Could not find include file 'sqlite3.h' in standard include directories. Defaulting to: '/usr/include/sqlite3.h' INC: sqlite3.h -> /usr/include/sqlite3.h LIB: libexpat.so -> /lib64 INC: expat.h -> /usr/include WARNING: Could not find library file 'libnetcdf.so' in standard library directories. Defaulting to: '/usr/lib/libnetcdf.so' LIB: libnetcdf.so -> /usr/lib

WARNING: Could not find include file 'netcdf.h' in standard include directories. Defaulting to: '/usr/include/netcdf.h' INC: netcdf.h -> /usr/include/netcdf.h Building with: CC: gcc CXX: g++ CFLAGS: -I/usr/include CXXFLAGS: -I/usr/include LDFLAGS: -L/usr/lib64 -Wl,--no-as-needed,-rpath-link,/usr/lib64 Setting environment variables Deleting previous plugin directory /vmd-1.9.3/vmd-python/build/lib/vmd/plugins Compiling plugins to /vmd-1.9.3/vmd-python/build/lib/vmd/plugins Building Molecule File Reader plugins /usr/bin/ld: cannot find crti.o: No such file or directory /usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libstdc++.so when searching for -lstdc++ /usr/bin/ld: cannot find -lstdc++ /usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc.a when searching for -lgcc /usr/bin/ld: cannot find -lgcc /usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc_s.so when searching for -lgcc_s /usr/bin/ld: cannot find -lgcc_s /usr/bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc /usr/bin/ld: skipping incompatible /usr/lib64/libc.a when searching for -lc /usr/bin/ld: skipping incompatible //usr/lib64/libc.so when searching for -lc /usr/bin/ld: skipping incompatible //usr/lib64/libc.a when searching for -lc /usr/bin/ld: cannot find -lc /usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc.a when searching for -lgcc /usr/bin/ld: cannot find -lgcc /usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc_s.so when searching for -lgcc_s /usr/bin/ld: cannot find -lgcc_s /usr/bin/ld: cannot find crtn.o: No such file or directory collect2: error: ld returned 1 exit status make[2]: [abinitplugin.so] Error 1 make[1]: [molfilelibs] Error 1 make: ** [LINUX] Error 2 Traceback (most recent call last): File "setup.py", line 377, in 'test': VMDTest, File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "setup.py", line 39, in run self.execute(self.compile_vmd, [], msg="Compiling VMD") File "/usr/lib64/python2.7/distutils/cmd.py", line 349, in execute util.execute(func, args, msg, dry_run=self.dry_run) File "/usr/lib64/python2.7/distutils/util.py", line 324, in execute func(args) File "setup.py", line 59, in compile_vmd check_call(cmd, cwd=srcdir) File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/vmd-1.9.3/vmd-python/vmd/install.sh', 'LINUX', '/vmd-1.9.3/vmd-python/build/lib/vmd', '/usr']' returned non-zero exit status 2 [root@234bc51125a5 vmd-python]#

Eigenstate commented 5 years ago

I'll add a check for Power9 systems in setup.py and put the right logic back in the configure script which hopefully will fix your problems with plugins not building. What's the output of: python -c "import platform; print(platfom.system(), platform.machine()"?

austinpagan commented 5 years ago

You had some typos, but here's what I think you want:

$ python -c "import platform; print(platform.system(), platform.machine())" ('Linux', 'ppc64le') $

austinpagan commented 5 years ago

Just FYI, I would be very happy to act as an alpha test site, as you work to get this running on Power architectures. I will not be at all disappointed if this becomes an iterative process over several days.

austinpagan commented 5 years ago

Also, a question: I see that your implementation supports offscreen buffers (what I have come to know as "openglpbuffer"). My belief is that a prerequisite for painting to such pbuffers is the use of EGL in VMD, but I'm not sure that is true. In any case, does your VMD-as-a-Python-library have the option to use EGL?

Eigenstate commented 5 years ago

Alright, I've made a new branch, ppc64le with an initial attempt. The build logic for this architecture seems to be pretty much identical except for no linkage to libstdc++ (hence the errors you were seeing earlier). Let me know if it works!

I am in the middle of polishing a new release and that does include support for offscreen rendering with EGL if you build with python setup.py build --egl but this is pretty experimental for now. You'll need the EGL development headers and (EGL/egl.h) and the OpenGL entry point library (libOpenGL.so) for the build to succeed.

austinpagan commented 5 years ago

So, I did these things:

git clone --branch ppc64le https://github.com/Eigenstate/vmd-python pip install --upgrade pip pip install --trusted-host pypi.python.orig numpy cd vmd-python python setup.py build

The last command generated this output:

[vmd-python]# python setup.py build running build running build_py package init file 'vmd/init.py' not found (or not a regular file) package init file 'vmd/init.py' not found (or not a regular file) Compiling VMD Finding libraries... LIB: libtcl8.5.so -> /usr/lib64/ INC: tcl.h -> /usr/include/ error: None [vmd-python]#

I will now attempt the "python setup.py install" and the next comment will contain those results.

austinpagan commented 5 years ago

heh.

"python setup.py install" gave me almost the exact same output as "python setup.py build":

[vmd-python]# python setup.py install running install running build running build_py package init file 'vmd/init.py' not found (or not a regular file) package init file 'vmd/init.py' not found (or not a regular file) Compiling VMD Finding libraries... LIB: libtcl8.5.so -> /usr/lib64/ INC: tcl.h -> /usr/include/ error: None [vmd-python]#

Maybe that missing init.py file is actually something important?

austinpagan commented 5 years ago

For future reference, can you point me to the EGL/egl.h and the libOpenGL.so?

Eigenstate commented 5 years ago

Hmmmm, that's weird. It's not the init.py, that's a benign error as that file is moved from elsewhere during installation. Although the output is missing it probably is failing to find libsqlite3. I've committed some better error messages. What distro are you running?

On Debian, you'll need the following packages for the dependencies:

austinpagan commented 5 years ago

Here's what I see when I type "cat /etc/os-release":

NAME="CentOS Linux" VERSION="7 (AltArch)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (AltArch)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7:server" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"

austinpagan commented 5 years ago

So, I tried on a simpler P9 system, with fewer bells and whistles, and it did give me more debug info:

[fossum@fstgbfen1 ~]$ git clone --branch ppc64le https://github.com/Eigenstate/vmd-python Cloning into 'vmd-python'... remote: Enumerating objects: 46, done. remote: Counting objects: 100% (46/46), done. remote: Compressing objects: 100% (32/32), done. remote: Total 9038 (delta 25), reused 30 (delta 14), pack-reused 8992 Receiving objects: 100% (9038/9038), 39.11 MiB | 14.21 MiB/s, done. Resolving deltas: 100% (3733/3733), done. Checking out files: 100% (2498/2498), done. [fossum@fstgbfen1 ~]$ cd vmd-python [fossum@fstgbfen1 vmd-python]$ python setup.py build running build running build_py package init file 'vmd/init.py' not found (or not a regular file) package init file 'vmd/init.py' not found (or not a regular file) Compiling VMD Finding libraries... LIB: libtcl8.5.so -> /usr/lib64/ INC: tcl.h -> /usr/include/ Traceback (most recent call last): File "setup.py", line 407, in 'test': VMDTest, File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "setup.py", line 66, in run self.execute(self.compile_vmd, [], msg="Compiling VMD") File "/usr/lib64/python2.7/distutils/cmd.py", line 349, in execute util.execute(func, args, msg, dry_run=self.dry_run) File "/usr/lib64/python2.7/distutils/util.py", line 324, in execute func(*args) File "setup.py", line 77, in compile_vmd self.set_environment_variables(pydir) File "setup.py", line 284, in set_environment_variables os.environ["SQLITELIB"] = "-L%s" % self._find_library_dir("libsqlite3") File "setup.py", line 201, in _find_library_dir libfile); RuntimeError: Could not find library 'libsqlite3.so' in standard library directories. Update $LD_LIBRARY_PATH to include the directory containing this file, or make sure it is present on your system [fossum@fstgbfen1 vmd-python]$

On this system, when I type "cat /etc/os-release" I get this:

[fossum@fstgbfen1 vmd-python]$ cat /etc/os-release NAME="Red Hat Enterprise Linux Server" VERSION="7.5 (Maipo)" ID="rhel" ID_LIKE="fedora" VARIANT="Server" VARIANT_ID="server" VERSION_ID="7.5" PRETTY_NAME="Red Hat Enterprise Linux Server 7.5 (Maipo)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:7.5:GA:server" HOME_URL="https://www.redhat.com/" BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" REDHAT_BUGZILLA_PRODUCT_VERSION=7.5 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="7.5" [fossum@fstgbfen1 vmd-python]$

austinpagan commented 5 years ago

I searched for libsqlite3.so in my /usr directory, and it is not there. Should I be doing a 'yum install' of one or more of the following:

sqlite-devel.ppc64le : Development tools for the sqlite3 embeddable SQL database sqlite-doc.noarch : Documentation for sqlite sqlite-tcl.ppc64le : Tcl module for the sqlite3 embeddable SQL database engine sqlite2-devel.ppc64le : Development files for SQLite sqlite2-tcl.ppc64le : Tcl bindings for sqlite sqlite3-dbf.ppc64le : Converter of XBase / FoxPro tables to SQLite sqlite.ppc64le : Library that implements an embeddable SQL database engine sqlite2.ppc64le : Embeddable SQL engine in a C library

?

austinpagan commented 5 years ago

Progress! I noodled around and concluded that installing the following three libraries got me to a place where I'm actually doing non-trivial things in "python setup.py build":

yum install sqlite-devel.ppc64le yum install netcdf.ppc64le yum install netcdf-devel.ppc64le

austinpagan commented 5 years ago

"python setup.py build" finished. "python setup.py install" finished. I saved the text of both steps in files if you want to look them over in detail.

The last few lines of the install step looked like this:

byte-compiling /usr/lib/python2.7/site-packages/vmd/test/test_material.py to test_material.pyc byte-compiling /usr/lib/python2.7/site-packages/vmd/test/test_molecule.py to test_molecule.pyc byte-compiling /usr/lib/python2.7/site-packages/vmd/test/test_selection.py to test_selection.pyc byte-compiling /usr/lib/python2.7/site-packages/vmd/init.py to init.pyc byte-compiling /usr/lib/python2.7/site-packages/vmd/Molecule.py to Molecule.pyc byte-compiling /usr/lib/python2.7/site-packages/vmd/Label.py to Label.pyc byte-compiling /usr/lib/python2.7/site-packages/vmd/Material.py to Material.pyc running install_egg_info Writing /usr/lib/python2.7/site-packages/vmd_python-3.0.0-py2.7.egg-info

Does that look good?

jvermaas commented 5 years ago

I think that is normal. The best test is of course to write a short python script. Something like: import vmd in an interactive or batch python session should now work, and help vmd should too.

austinpagan commented 5 years ago

OK, so I started an interactive session, and this happened:

[root@f52816e1f650 vmd-1.9.3]# python Python 2.7.5 (default, Oct 30 2018, 23:44:29) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

import vmd Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/site-packages/vmd/init.py", line 25, in vmd = imp.load_dynamic(name, vmdlib) ImportError: /usr/lib/python2.7/site-packages/vmd/libvmd.so: cannot dynamically load executable

And here's what's going on with that library:

[root@f52816e1f650 vmd-1.9.3]# ls -lt /usr/lib/python2.7/site-packages/vmd/*.so -rwxr-xr-x 1 root root 6198512 Feb 27 18:46 /usr/lib/python2.7/site-packages/vmd/libvmd.so [root@f52816e1f650 vmd-1.9.3]#

And (perhaps unsurprisingly) when I tried "help vmd" I got this:

[root@f52816e1f650 vmd-1.9.3]# python Python 2.7.5 (default, Oct 30 2018, 23:44:29) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

help vmd File "", line 1 help vmd ^ SyntaxError: invalid syntax

jvermaas commented 5 years ago

Ok, serves me right for not testing things. Shoulda been help(vmd), but that isn't the problem. I think the .so wasn't compiled as a dynamically loaded executable, since it was missing the PIC argument during compilation. I've got a pull request open. Does making those changes fix the problem?

austinpagan commented 5 years ago

I'm stupid about doing things with "git". Should I just do another git clone --branch ppc64le https://github.com/Eigenstate/vmd-python and then re-do the builds?

austinpagan commented 5 years ago

I'm going to take a break. I'll be back here around 7PM Central Time, and maybe the "git pull request" will be pulled into the main line, or whatever y'all do to synchronize these things, and I can do a fresh git clone?

jvermaas commented 5 years ago

Sure. You can already do a clone off of my fork, it just isn't in the mainline yet. git clone --branch ppc64le git@github.com:jvermaas/vmd-python.git

austinpagan commented 5 years ago

[root@f52816e1f650 vmd-1.9.3]# git clone --branch ppc64le git@github.com:jvermaas/vmd-python.git Cloning into 'vmd-python'... Permission denied (publickey). fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists. [root@f52816e1f650 vmd-1.9.3]#

jvermaas commented 5 years ago

Weird. I assume the web version shows up fine? https://github.com/jvermaas/vmd-python I know I have my ssh keys setup, but it looks like above you use the https version instead, so that might be the issue.

austinpagan commented 5 years ago

Yes, thank you! I now am trying to build and install your version. More info soon.

Eigenstate commented 5 years ago

I merged @jvermaas pull request so you can also try the ppc64le branch again

austinpagan commented 5 years ago

OK, so that's different...

I did the following with BOTH of your builds: (a) rm -rf vmd-python (b) git clone --branch ppc64le https://github.com/_{name}_/vmd-python (c) cd vmd-python (d) python setup.py build (e) python setup.py install (f) python (g) >>> import vmd

In both cases, the resulting errors were as follows:

Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/xsfplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/xsfplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/pbeqplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/pbeqplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/graspplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/graspplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/dmsplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/dmsplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/parmplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/parmplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/spiderplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/spiderplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/molemeshplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/molemeshplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/dtrplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/dtrplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/brixplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/brixplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/parm7plugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/parm7plugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/fs4plugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/fs4plugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/maeffplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/maeffplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/raster3dplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/raster3dplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/ccp4plugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/ccp4plugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/msmsplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/msmsplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/plyplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/plyplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/pltplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/pltplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/situsplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/situsplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/biomoccaplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/biomoccaplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/amiraplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/amiraplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/vtkplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/vtkplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/edmplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/edmplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/gromacsplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/gromacsplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/phiplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/phiplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/grdplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/grdplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/pdbxplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/pdbxplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/avsplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/avsplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/mdfplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/mdfplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/cubeplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/cubeplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/dsn6plugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/dsn6plugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/gridplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/gridplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/mapplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/mapplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/dxplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/dxplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/stlplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/stlplugin.so Warning) Unable to dlopen '/usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/uhbdplugin.so': Unable to dlopen plugin file /usr/lib/python2.7/site-packages/vmd/plugins/LINUXPPC64/molfile/uhbdplugin.so

austinpagan commented 5 years ago

I am but an egg, but I'm guessing the plugins need the same treatment you gave the main library?

jvermaas commented 5 years ago

PROGRESS! You can choose to ignore these errors, unless you happen to need to load a file that these plugins are there to handle. I suspect its the same sort of issue with -fPIC not getting called, since typically these get compiled into the executable in a stand-alone VMD build.

austinpagan commented 5 years ago

Well, when I try "help(vmd)" I get an error:

help(vmd) Traceback (most recent call last): File "", line 1, in NameError: name 'vmd' is not defined

austinpagan commented 5 years ago

Maybe you can suggest a "hello world" program I can try to see if vmd is actually working?

austinpagan commented 5 years ago

And maybe there's no defined "help screen" for vmd yet?

austinpagan commented 5 years ago

ERASE ERASE ERASE I was not aware that you have to import a package before you can call help on the package. "help(vmd)" actually returns useful stuff!

jvermaas commented 5 years ago 
from vmd import evaltcl
evaltcl('vmdinfo version')

Would be the basic test. Glad you got the import sorted. :)

austinpagan commented 5 years ago

from vmd import evaltcl evaltcl('vmdinfo version') '1.9.4a26'

austinpagan commented 5 years ago

So, moving on to the next phase: @Eigenstate ... my manager is going to see your comments that EGL is supported and he's going to wonder why I can't just do the build specifying the "--egl" flag, as you indicated in your README. But I'm thinking that flag is only valid on Debian systems? When I tried it I got an error.

Eigenstate commented 5 years ago

What's the error message? You'll need to install the header file and library dependencies for EGL in order for the build to succeed:

On Centos, you can use yum whatprovides or similar to find which package names provide these dependencies. The naming can differ between distros so I only list the packages for Debian which is what I run personally.

austinpagan commented 5 years ago

My manager went to a lot of effort to ensure that EGL was enabled on this box:

[root@f52816e1f650 vmd-python]# ls /usr/local/include/GL/gl.h /usr/local/include/GL/gl.h [root@f52816e1f650 vmd-python]# ls /usr/local/include/EGL/ /usr/local/include/EGL/eglext.h /usr/local/include/EGL/egl.h /usr/local/include/EGL/eglplatform.h [root@f52816e1f650 vmd-python]# ls /lib64/libOpenGL /lib64/libOpenGL.so /lib64/libOpenGL.so.0 /lib64/libOpenGL.so.0.0.0 [root@f52816e1f650 vmd-python]#

Here's the error I get:

[root@f52816e1f650 vmd-1.9.3]# rm -rf vmd-python [root@f52816e1f650 vmd-1.9.3]# git clone --branch ppc64le https://github.com/Eigenstate/vmd-python Cloning into 'vmd-python'... remote: Enumerating objects: 86, done. remote: Counting objects: 100% (86/86), done. remote: Compressing objects: 100% (59/59), done. remote: Total 9078 (delta 52), reused 53 (delta 27), pack-reused 8992 Receiving objects: 100% (9078/9078), 39.15 MiB | 19.04 MiB/s, done. Resolving deltas: 100% (3760/3760), done. [root@f52816e1f650 vmd-1.9.3]# cd vmd-python [root@f52816e1f650 vmd-python]# python setup.py --egl build usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help

error: option --egl not recognized [root@f52816e1f650 vmd-python]#

jvermaas commented 5 years ago

Try it the other way around. python setup.py build --egl

austinpagan commented 5 years ago

It's compiling. I suppose if the requisite OpenGL and EGL include/libs are in bad directories, I can get past that with some environment variables?

austinpagan commented 5 years ago

echo "Linking " libvmd.so "..."; \ cd ../LINUXPPC64 ; \ g++ -m64 -Wall -Wno-unknown-pragmas -O3 -ffast-math -fPIC -DVMD_SHARED -DARCH_LINUXPPC64 -DVMDOPENGLPBUFFER -DVMDEGLPBUFFER -DVMDCOLVARS -DVMDIMD -DVMDPYTHON -DVMDTHREADS -DWKFTHREADS -DUSEPOSIXTHREADS -D_REENTRANT -DVMDNUMPY -DVMDQUICKSURF -DVMDWITHCARBS -DVMDPOLYHEDRA -DVMDSURF -DVMDMSMS -DVMDNANOSHAPER -DVMDLATTICECUBES -DVMDTCL -DVMDSTATICPLUGINS -I/usr/include/python2.7 -I/usr/lib64/python2.7/site-packages/numpy/core/include -I/usr/lib64/python2.7/site-packages/numpy/core/lib -I/usr/include/ -I../plugins/include -I../plugins/LINUXPPC64/molfile -I/usr/include/ -I. -I../src -o libvmd.so OpenGLPbufferDisplayDevice.o OpenGLExtensions.o OpenGLRenderer.o OpenGLShader.o OpenGLCache.o colvaratoms.o colvarbias.o colvarbias_abf.o colvarbias_alb.o colvarbias_histogram.o colvarbias_meta.o colvarbias_restraint.o colvar.o colvarcomp.o colvarcomp_angles.o colvarcomp_coordnums.o colvarcomp_distances.o colvarcomp_protein.o colvarcomp_rotations.o colvardeps.o colvargrid.o colvarmodule.o colvarparse.o colvarproxy.o colvarproxy_vmd.o colvarscript.o colvartypes.o colvarvalue.o IMDMgr.o IMDSim.o IMDSimThread.o CmdIMD.o imd.o PythonTextInterp.o VMDTkinterMenu.o py_animate.o py_atomsel.o py_axes.o py_color.o py_commands.o py_display.o py_graphics.o py_imd.o py_label.o py_material.o py_measure.o py_menu.o py_molecule.o py_molrep.o py_mouse.o py_render.o py_selection.o py_topo.o py_trans.o py_vmd.o py_numeric.o androidvmdstart.o Animation.o ArtDisplayDevice.o AtomColor.o AtomParser.o AtomLexer.o AtomRep.o AtomSel.o Axes.o BaseMolecule.o Benchmark.o BondSearch.o CmdAnimate.o CmdColor.o CmdDisplay.o CmdLabel.o CmdMaterial.o CmdMenu.o CmdMol.o CmdRender.o CmdTrans.o CommandQueue.o CoorPluginData.o CUDAAccel.o DisplayDevice.o Displayable.o DisplayRocker.o DispCmds.o DrawMolecule.o DrawMolItem.o DrawMolItem2.o DrawMolItemRibbons.o DrawMolItemMSMS.o DrawMolItemNanoShaper.o DrawMolItemRings.o DrawMolItemOrbital.o DrawMolItemQuickSurf.o DrawMolItemSurface.o DrawMolItemVolume.o DrawForce.o DrawRingsUtils.o FastPBC.o FileRenderList.o FileRenderer.o FPS.o GaussianBlur.o GelatoDisplayDevice.o GeometryAngle.o GeometryAtom.o GeometryBond.o GeometryDihedral.o GeometryList.o GeometryMol.o GeometrySpring.o Hershey.o HMDMgr.o Inform.o ImageIO.o Isosurface.o JRegex.o JString.o macosxvmdstart.o MaterialList.o Matrix4.o MayaDisplayDevice.o MDFF.o Measure.o MeasureCluster.o MeasurePBC.o MeasureRDF.o MeasureQCP.o MeasureSurface.o MeasureSymmetry.o MeasureVolInterior.o MobileInterface.o MobileButtons.o MobileTracker.o Molecule.o MoleculeList.o MoleculeGraphics.o MolFilePlugin.o Mouse.o MSMSInterface.o NanoShaperInterface.o Orbital.o OrbitalJIT.o PeriodicTable.o P_JoystickTool.o P_TugTool.o P_GrabTool.o P_PrintTool.o P_PinchTool.o P_UIVR.o P_Buttons.o P_Tracker.o P_Tool.o P_CmdTool.o P_SensorConfig.o ParseTree.o PickList.o PickModeAddBond.o PickModeCenter.o PickModeForce.o PickModeList.o PickModeMolLabel.o PickModeMove.o PickModeUser.o PlainTextInterp.o PluginMgr.o POV3DisplayDevice.o PSDisplayDevice.o QMData.o QMTimestep.o QuickSurf.o RadianceDisplayDevice.o RayShadeDisplayDevice.o R3dDisplayDevice.o RenderManDisplayDevice.o Scene.o ScaleSpaceFilter.o Segmentation.o SnapshotDisplayDevice.o Spaceball.o SpaceballButtons.o SpaceballTracker.o SpatialSearch.o SpringTool.o Stage.o STLDisplayDevice.o Stride.o Surf.o SymbolTable.o TachyonDisplayDevice.o Timestep.o UIObject.o UIText.o VideoStream.o VMDApp.o VMDCollab.o VMDDir.o VMDDisplayList.o VMDMenu.o VMDQuat.o VMDTitle.o VMDThreads.o VolCPotential.o VolMapCreate.o VolMapCreateILS.o VolumetricData.o VolumeTexture.o Voltool.o VrmlDisplayDevice.o Vrml2DisplayDevice.o Watershed.o WavefrontDisplayDevice.o WKFThreads.o WKFUtils.o utilities.o util_simd.o vmd.o vmdmain.o X3DDisplayDevice.o ColorInfo.o TclCommands.o TclFastPBC.o TclMDFF.o TclMeasure.o TclMolInfo.o TclTextInterp.o TclVec.o TclGraphics.o TclSegmentation.o TclVolMap.o TclVoltool.o cmd_animate.o cmd_collab.o cmd_color.o cmd_display.o cmd_imd.o cmd_label.o cmd_material.o cmd_menu.o cmd_mobile.o cmd_mol.o cmd_mouse.o cmd_parallel.o cmd_plugin.o cmd_profile.o cmd_render.o cmd_spaceball.o cmd_tool.o cmd_trans.o cmd_user.o cmd_util.o cmd_videostream.o cmd_vmdbench.o tcl_commands.o VMDTkMenu.o hash.o inthash.o intstack.o ptrstack.o msmpot.o msmpot_compute.o msmpot_cubic.o msmpot_setup.o vmdsock.o vmddlopen.o pcre.o fitrms.o -L/usr/lib/python2.7/site-packages -L/usr/lib64/ -L../plugins/LINUXPPC64/molfile -L/usr/lib64/ -lEGL -L/usr/X11R6/lib -lGL -lXext -lX11 -lpthread -ltcl8.5 -lmolfile_plugin -lnetcdf -lm -ldl -lsqlite3 -lexpat -L/usr/lib64 -lpython2.7 -Wl,-rpath,/usr/X11R6/lib -Wl,-rpath,/usr/local/lib -shared ; Linking libvmd.so ... /usr/bin/ld: cannot find -lGL /usr/bin/ld: cannot find -lXext collect2: error: ld returned 1 exit status

Eigenstate commented 5 years ago

Looks like the build is pulling in the wrong libraries. Let me take a look.

austinpagan commented 5 years ago

Here in flyover country it's nearly 9PM, so I'll check back here at 7AM tomorrow (5AM your time). Thanks SO MUCH for enabling this functionality for us!

austinpagan commented 5 years ago

Just to be clear, we believe (based on your README) that EGL is a prerequisite for using offscreen "pbuffers," and we believe that pbuffers are pretty much mandatory for visualization in any environment that doesn't have a local windowing system (like large clusters of servers). Is that correct? Because if not, we are open to other options...

jvermaas commented 5 years ago

Just curious, but what happens when you do the following:

from vmd import render
print render.listall()

VMD has plenty of rendering capabilities that depend on external programs. In principle you could use the render pipeline and those external programs to make images if you choose, and I don't think that those require OpenGL at all, although I may be quite mistaken.

austinpagan commented 5 years ago

from vmd import render print render.listall() ['ART', 'Gelato', 'PostScript', 'Raster3D', 'Radiance', 'Rayshade', 'RenderMan', 'snapshot', 'STL', 'Tachyon', 'POV3', 'VRML-1', 'VRML-2', 'Wavefront', 'X3D', 'X3DOM']

austinpagan commented 5 years ago

I had been using "snapshot" to capture things stored in pbuffers and output them to a .tga file. One program I had recently done (using vmd directly) looked like this:

vmd -dispdev openglpbuffer vmd> mol new proteins/alanin.pdb vmd> graphics 0 cylinder {1 1 1} {2 2 2} radius 2 resolution 60 filled no vmd> graphics 0 color 3 vmd> graphics 0 cylinder {3 3 3} {4 4 4} radius 2 resolution 60 filled no vmd> render snapshot firstfile.tga

jvermaas commented 5 years ago

Weird. Even with -dispdev text normal VMD can render just fine, but I tried it myself and vmd-python can't quite do that the way I compiled it (without egl stuff). So it looks like that option has a purpose after all.

austinpagan commented 5 years ago

I inquired of my manager, regarding the reasoning behind our focus on EGL. He told me this, which I share with you to help explain our understanding of the importance of EGL:

Here is the general strategy: If running on client system, then use whatever window system is appropriate for your client, otherwise (on a server) use EGL. EGL is not a replacement for OpenGL. It is just a way to initialize the GPU, select a visual, etc without the heavyweight and non-secure X-server.

austinpagan commented 5 years ago

I'm not in a hurry here, and I'm sure you have many things that are higher priority, but could I get a warm fuzzy that you're still hoping to provide an update for me to try?