Work impossible with start_annotald

[diertani]

T-0: Annotald invoked with start_annotald. T+2 minutes: Annotald crashes. T+3 minutes: Annotald invoked with start_annotald. T+3 minutes: Annotald crashes. T+4 minutes: Annotald invoked with start_annotald. T+5 minutes: Annotald crashes. T+6 minutes: Annotald invoked with start_annotald. T+6 minutes: Annotald crashes. T+7 minutes: Annotald invoked with annotald. T+17 minutes: Annotald closed without crashing. T+17 minutes: Annotald invoked with start_annotald.

Basically, I haven't been able to get any work done while using the start_annotald invocation (I have not been able to produce any savable work with the new script so far at all), but have not had problems at all (so far) with the older invocation. Is this supposed to happen when I use start_annotald, or is it possible that there's a bug in the new script that is causing Annotald to crash more than necessary?

For the moment, since I am completely unable to work while using the new command, I'm going to assume that it has a bug and return to using the original command. Let me know if this is an erroneous assumption and/or if there's anything else I should do instead.

[diertani]

This is what the start_annotald error message looks like:

127.0.0.1 - - [08/Sep/2014:08:42:44] "POST /doLogEvent HTTP/1.1" 200 - "http://localhost:1234/diertani" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.103 Safari/537.36" /home/diertani/bin/start_annotald: line 3: 25869 Segmentation fault strace -o $outfile -f /usr/local/bin/annotald -s /home/diertani/.local/bin/eng-settings.js $* babel /home/migration/other/MIDENG/PPCMBE/psd/stage-2/DIERTANI> channel 6: open failed: connect failed: Connection refused channel 3: open failed: connect failed: Connection refused channel 3: open failed: connect failed: Connection refused channel 3: open failed: connect failed: Connection refused

[aecay]

I think I see the cause of the error. Please remove all annotaldLog.shelve files from directories you work in. (Please do not delete the annotaldLog.txt files.) As a stopgap, that should cause a decrease in the number of errors.

I will code a proper fix and ask Vince to deploy the new version to babel in the next day or two.

[aecay]

That is, I see the cause of the error when running with start_annotald. I hope that it manifests itself more in that case because the debugging code in start_annotald puts the system under more pressure, and thus flushes out the underlying errors earlier. Once the fix is in place you will have to let me know if there are any further segmentation faults. Let's hope not...

[diertani]

Okay, annotaldLog.shelve has been removed. New experiment commencing.

[diertani]

We'd been doing great -- almost three hours with only two segmentation faults, using start_annotald. Deleting the .shelve file has definitely been a vast improvement over yesterday. Then I got a brand-new error message:

127.0.0.1 - - [09/Sep/2014:11:01:01] "POST /doLogEvent HTTP/1.1" 200 - "http://localhost:1234/diertani" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.103 Safari/537.36"
[09/Sep/2014:11:01:36] ENGINE Restarting because /usr/local/bin/annotald changed.
[09/Sep/2014:11:01:36] ENGINE Stopped thread 'Autoreloader'.
[09/Sep/2014:11:01:36] ENGINE Bus STOPPING
[09/Sep/2014:11:01:36] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('127.0.0.1', 10619)) shut down
[09/Sep/2014:11:01:36] ENGINE Stopped thread '_TimeoutMonitor'.
[09/Sep/2014:11:01:36] ENGINE Bus STOPPED
[09/Sep/2014:11:01:36] ENGINE Bus EXITING
[09/Sep/2014:11:01:36] ENGINE Bus EXITED
[09/Sep/2014:11:01:36] ENGINE Waiting for child threads to terminate...
[09/Sep/2014:11:01:36] ENGINE Re-spawning /usr/local/bin/annotald -s /home/diertani/.local/bin/eng-settings.js -p 10619 decourcy-1776.psd
PANIC: attached pid 11332 exited with 0
PANIC: handle_group_exit: 11332 leader 11330
Traceback (most recent call last):
  File "/usr/local/bin/annotald", line 14, in <module>
    import annotald.treedrawing
  File "/home/diertani/.local/lib/python2.7/site-packages/annotald/treedrawing.py", line 45, in <module>
    import nltk.tree as T
  File "/usr/local/lib/python2.7/site-packages/nltk/__init__.py", line 117, in <module>
ImportError: No module named align

It actually says "PANIC." I have no idea what that means, but it sounds very exciting.

[diertani]

Okay, so the error message in the previous comment is not the actual error message. It looks like I can't copy it perfectly -- everything in angled brackets is getting erased, and something weird is happening when there's an underscore followed by a period (that seems to be what caused the italics?). The gaps mostly are "module" in angled brackets.

[diertani]

Oh dear... I've killed it. Now for some reason I can't use either start_annotald or annotald anymore. I'm not doing anything differently with the invocation command, but I keep getting a 500 Internal Server Error.

Traceback (most recent call last):
  File "/home/diertani/.local/lib/python2.7/site-packages/cherrypy/_cprequest.py", line 670, in respond
    response.body = self.handler()
  File "/home/diertani/.local/lib/python2.7/site-packages/cherrypy/lib/encoding.py", line 217, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/home/diertani/.local/lib/python2.7/site-packages/cherrypy/_cpdispatch.py", line 61, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/home/diertani/.local/lib/python2.7/site-packages/annotald/treedrawing.py", line 406, in inner_index
    currentHtml = self.treesToHtml(currentTrees)
  File "/home/diertani/.local/lib/python2.7/site-packages/annotald/treedrawing.py", line 334, in treesToHtml
    nltk_tree = T.Tree(tree)
  File "/usr/local/lib/python2.7/site-packages/nltk/tree.py", line 99, in __init__
    % type(self).__name__)
TypeError: Tree: Expected a node value and child list

Got the same error message with the old command, so... eeish. I have no idea what I did.

[diertani]

Things I've tried so far:

1. Logging off babel and logging back in.
2. Deleting the new annotaldLog.shelve file that seems to have been generated.

Still getting the same error message. Let me know if there's anything else I should try.

[aecay]

Sigh. Two days ago, one of the other python packages that Annotald depends on was updated, and today Vince updated Annotald on babel, pulling in this bad update. I've made another minor version of Annotald to fix this, and I'll ask Vince to install it.

[aecay]

For future reference, you can make the error messages not do funny formatting things if you put three backticks (```) on a line by themselves both before and after the pasted text. I've edted your comments above to have this in them.

[aecay]

OK, this should now be fixed. And you can (should) stop using start_annotald and go back to just annotald as your command. The debugging output from the former is (hopefully) no longer necessary.