Ncl

[jimallman]

@mtholder I've merged this into my local main and it's working well on devtree, with one exception. ~It doesn't seem to modify the curation app's config file, so a curator still can't upload a Newick file since the app can't find the setting [external] > 2nexml~

UPDATE: Well now I'm really confused. I see that this is pre-baked into the "curatorful" config file, and it showed up (w/ a correct path) there the next time I deployed the webapps. :bowtie: Not sure what's changed, maybe it was previously deployed from a different branch of ot-ansible?

~I see how this setup was done in germinator, but this was done as part of installing the web apps. Now that NCL has its own playbook, I'm not sure how we should approach this. I assume NCL can be installed on its own, or in other contexts, right?~

~There are ways to nest playbooks, so maybe we should call playbk-install-ncl.yml from within playbk-install-webapp.yml, and modify the curation app's config only when it's installed? (Or we could test for the existence of the curation app and modify it when installing NCL, but it seems like an important dependency, so it's probably best to make sure NCL is included when installing the web apps.)~

~I suppose we could also just use include_role to install ncl in the webapp playbook, esp since {{ install_location }} and {{ repo_dir }} are already defined for both. But this would miss (or require duplication of) the needed apt packages in the ncl playbook. Thoughts?~

[jimallman]

While testing the config file above, I ran into a related error when uploading Newick from the curation app. Here's the traceback from web2py:

Traceback (most recent call last):
  File "/home/deploy/repos/web2py/gluon/restricted.py", line 219, in restricted
    exec(ccode, environment)
  File "/home/deploy/repos/web2py/applications/curator/controllers/default.py", line 771, in <module>
  File "/home/deploy/repos/web2py/gluon/globals.py", line 422, in <lambda>
    self._caller = lambda f: f()
  File "/home/deploy/repos/web2py/applications/curator/controllers/default.py", line 647, in to_nexson
    wait=block)
  File "applications/curator/modules/externalproc.py", line 225, in do_ext_proc_launch
    p = subprocess.Popen(wrapped_invoc)\n  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 13] Permission denied

This occurs when we finally try to invoke NCLconverter at {{ repo_dir }}/nclinstall/bin/NCLconverter. But when I check its permissions, it looks like this should have worked:

$ ls /home/deploy/repos/nclinstall/bin/NCLconverter -alF
-rwxr-xr-x 1 deploy deploy 1517768 Feb 28 03:41 /home/deploy/repos/nclinstall/bin/NCLconverter*

That is to say, anyone can execute NCLconverter here. Maybe the permission is on another file in this invocation (NEXML_FILENAME, ERR_FILENAME, something else)? I'll chase this, but postingΩ this in case it sparks an idea.

[jimallman]

Here's the full invocation (echoed to the logs):

WARNING: ***********************
WARNING: effective_user:
WARNING: deploy
WARNING: working_dir:
WARNING: /home/deploy/repos/ncl-scratch/2nexml/uaaf4fa83-cf84-4f0b-9b6d-a215612a6e46
WARNING: invoc:
WARNING: ['/home/deploy/repos/nclinstall/bin/NCLconverter', '-frelaxedphyliptree', '-X', '-x', '-g', '-pe557421', '-pO722600', '-pt92190', '-pT804707', '-po11747', '-pn202116', '-etreenexml', 'in.nex']
WARNING: NEXML_FILENAME:
WARNING: out.xml
WARNING: ERR_FILENAME:
WARNING: err.txt
WARNING: ***********************

Oddly, when I manually run exactly this (as user deploy) from the working directory, it works!

$ cd /home/deploy/repos/ncl-scratch/2nexml/uaaf4fa83-cf84-4f0b-9b6d-a215612a6e46
$ /home/deploy/repos/nclinstall/bin/NCLconverter -frelaxedphyliptree -X -x -g -pe502944 -pO760771 -pt601667 -pT99095 -po211527 -pn296384 -etreenexml in.nex
[Reading in.nex  ]
Executing
storing implied block: TAXA
storing read block: TREES
Writing out.xml

So I'm not sure why this always fails in normal use.