mrannotator crashes

[GoogleCodeExporter]

sept 15, 2009

I tried to run mrannotator on both grunt and tang. 

tang's run can't even open the entries. This is the error message:

ERROR: 
java.io.FileNotFoundException: 
/Users/jd/wattosTestingPlatform/Wattos/mr_anno_progress/2kn6.mr (No such file 
or directory)
    at java.io.FileOutputStream.open(Native Method)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:179)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:131)
    at Wattos.Utils.InOut.gunzipFile(InOut.java:412)
    at Wattos.Episode_II.MRAnnotate.annotateEntry(MRAnnotate.java:317)
    at Wattos.Episode_II.MRAnnotate.annotateLoop(MRAnnotate.java:244)
    at Wattos.Episode_II.MRAnnotate.main(MRAnnotate.java:795)

ERROR: Found throwable error above
ERROR: in MRAnnotate.annotateEntry found:
ERROR: Gunzipping the MR file failed.
ERROR: Skipping this entry.

grunt allows editing and will put the entry into "in progress," but then 
crashes mrannotator:

Doing: splitToBlocks
Number of lines in block: 30
Number of lines in block: 278
Number of lines in block: 13881
Checks successful.
Moving the MR file to the database.
ERROR: Failed to create containing dir if needed for dbfs file: 
/big/jurgen/DB/mrgrid/bfiles/wattos1/144/144470.dat
ERROR: in SQL_Episode_II.putMRFile found. Will rollback changes.
WARNING: Transaction is being rolled back
ERROR: rollback changes completed.
ERROR: Files might still reside in the DBFS without being referenced.
ERROR: It's suggested to clean the DBFS by querying the database for any unused 
files.
Retrieved number of files in db            : 142223
ERROR: Given File instance isn't a directory.
ERROR: Failed to InOut.getFilesRecursive from dir: 
/big/jurgen/DB/mrgrid/bfiles/wattos1
Exception in thread "main" java.lang.NullPointerException
    at Wattos.Episode_II.SQL_Episode_II.makeDBFSConsistent(SQL_Episode_II.java:1315)
    at Wattos.Episode_II.SQL_Episode_II.putMRFile(SQL_Episode_II.java:1238)
    at Wattos.Episode_II.DBMRFile.writeToDB(DBMRFile.java:237)
    at Wattos.Episode_II.MRAnnotate.annotateEntry(MRAnnotate.java:403)
    at Wattos.Episode_II.MRAnnotate.annotateLoop(MRAnnotate.java:244)
    at Wattos.Episode_II.MRAnnotate.main(MRAnnotate.java:795)

Original issue reported on code.google.com by schulte....@gmail.com on 15 Sep 2009 at 2:56

[GoogleCodeExporter]

I need one more day before I have tang up hopefully.
I send out a message to you and Jon before that I was working on this. I 
haven't finished yet.

Original comment by jurge...@gmail.com on 15 Sep 2009 at 3:00

[GoogleCodeExporter]

Did anything change on grunt or the archive? That's why I've included Dimitri 
in the cc.

Original comment by schulte....@gmail.com on 15 Sep 2009 at 3:03

[GoogleCodeExporter]

Why is grunt suddenly wanting to access /big/jurgen/DB/mrgrid/bfiles/wattos1? 
Where is this set?

Original comment by schulte....@gmail.com on 15 Sep 2009 at 3:12

[GoogleCodeExporter]

It looks like dbfs_dir is being set in /home/docr/wattos.runtime.properties.

I'm not going to touch that until you're done, but could you set a conditional 
to check for tang/grunt?

Original comment by schulte....@gmail.com on 15 Sep 2009 at 3:27

[GoogleCodeExporter]

Can you ask Dmitri to fix this administrative stuff? He developed this part.

Original comment by jurge...@gmail.com on 15 Sep 2009 at 3:41

[GoogleCodeExporter]

I'll talk to him about it, but it looks like wattos.runtime.properties was 
altered yesterday, and I don't think he 
would have touched that.

Original comment by schulte....@gmail.com on 15 Sep 2009 at 3:47

[GoogleCodeExporter]

I didn't. Why did you change things on grunt anyway? -- I thought the plan was 
to not
touch it until the changes have been debugged and tested on tang.

Original comment by dmitri.m...@gmail.com on 15 Sep 2009 at 4:35

[GoogleCodeExporter]

sorry i added the file for docr on tang
probably best to use a different loc such as /tmp
home is the same

Original comment by jurge...@gmail.com on 15 Sep 2009 at 4:42

[GoogleCodeExporter]

To prevent different setups of NRG on different machines from interacting with 
each other the home directory 
should not be shared. Perhaps on grunt you can setup a different user? 
Otherwise I'm afraid you would be 
mixing production, development, and slave setups.

The setup on tang works again; fixing issue 225.
From the NLs I was unable to easily reannotate the entry; Chris could you try 
that?

I leave this issue open for D.

Original comment by jurge...@gmail.com on 16 Sep 2009 at 8:55

[GoogleCodeExporter]

Wim updated ccpn on tang. 

I reran mrannotator and got

STDERR>Unable to access jarfile 
/Applications/jEdit.app/Contents/Resources/Java/jedit.jar
ERROR: in MRAnnotate.annotateEntry found:
ERROR: Error editing this entry. Please remove temporary files
ERROR: Skipping this entry.

Original comment by schulte....@gmail.com on 16 Sep 2009 at 2:41

[GoogleCodeExporter]

Go ahead and add a setting in ~docr/wattos.runtime.properties for:
editor                         : java -Xmx96m -Xms24m -jar 
/Applications/jEdit.app/Contents/Resources/Java/jedit.jar
to point to your favorite.

Original comment by jurge...@gmail.com on 16 Sep 2009 at 2:51

[GoogleCodeExporter]

Re: #9 -- I'm going to cut tang off the shared /home (and perhaps trim 
its /etc/passwd to only include Wim, Jurgen, and Chris).

I'll make copies of homedirs, you'll still have /share to move stuff between 
the 
machines.

Original comment by dmitri.m...@gmail.com on 16 Sep 2009 at 2:58

[GoogleCodeExporter]

Ok, just let me know before you do where the new home of docr@tang will be 
stationed.
Cheers

Original comment by jurge...@gmail.com on 16 Sep 2009 at 3:06

• Changed state: Fixed

[GoogleCodeExporter]

It will be in /home, but /home will be local. Can you get off tang so I can do 
it
without killing your login session?

Also, the easier way out is to define e.g. "wattos.config" property and pass it 
on
command line like this:
 java -Dwattos.config=/somewhere/somefile ... 
but that requires another recompile and reinstall.

Original comment by dmitri.m...@gmail.com on 16 Sep 2009 at 4:15

[GoogleCodeExporter]

Let's stick with your first solution of having a different /home. For the 
future use the mechanism with 
WATTOSHOME env. var..

Original comment by jurge...@gmail.com on 16 Sep 2009 at 9:04

[GoogleCodeExporter]

tang:/home is now on local disk on tang. Old homedirs are mounted in 
/mnt/oldhome.
I copied over home directories for cfschulte, docr, and jurgen users.
I had to shut down tomcat and kill server checking cron jobs while they were 
copying.

Note that home directories on grunt are as before: shared on all BMRB computers.

Original comment by dmitri.m...@gmail.com on 16 Sep 2009 at 9:54

[GoogleCodeExporter]

Not fixed on tang yet. Mrannotator is looking for jedit in the Mac's filePath 
(Things work on grunt, though):

editor                         : java -Xmx96m -Xms24m -jar 
/Applications/jEdit.app/Contents/Resources/Java/jedit.jar

TDERR>Unable to access jarfile 
/Applications/jEdit.app/Contents/Resources/Java/jedit.jar
ERROR: in MRAnnotate.annotateEntry found:
ERROR: Error editing this entry. Please remove temporary files
ERROR: Skipping this entry.
Continue annotation series? (y/n): n
Start over? (y/n): n
/

Original comment by schulte....@gmail.com on 17 Sep 2009 at 5:53

• Changed state: Started

[GoogleCodeExporter]

See comment 11. You need to add the setting to the 
~docr/wattos.runtime.properties file

Original comment by jurge...@gmail.com on 17 Sep 2009 at 6:04

[GoogleCodeExporter]

Sorry, I didn't register on the last post. I was focused on something else. 
I'll check the properties file also to 
understand what's going on.

I found the conditional that was messing up in Episode_II/Globals.java on line 
207.

changed:
            if( hostname.equals("grunt") ) {
 to 
            if( hostname.equals("grunt") || hostname.equals("tang")) {

We have way too many files where paths and properties are set.

Original comment by schulte....@gmail.com on 17 Sep 2009 at 8:26

• Changed state: Fixed

[GoogleCodeExporter]

No problem. Easiest way I think is to focus on the file settings and forget all 
about the code settings. If it's 
broken fix it in the local file.

Original comment by jurge...@gmail.com on 17 Sep 2009 at 8:29

[GoogleCodeExporter]

Yes. A local config file can make things easier in the long run.

Original comment by schulte....@gmail.com on 17 Sep 2009 at 9:12