Need a service to provide lineage and incertae sedis information for a taxon

[jar398]

There should be a way to look at the lineage, along with the incertae sedis status of any link in the lineage. (Maybe all flags, I don't know - at least the ones related to taxa included by taxomachine and excluded by treemachine, such as extinct.)

One application would be to help explain why a taxon is excluded from synthesis. See https://github.com/OpenTreeOfLife/opentree/issues/310 .

Another application is automated regression testing for the taxonomy, where the main question is "is A under B?"

[chinchliff]

Ok. I added a boolean option to the getTaxonInfo service. This feature is functioning properly in commit 35fb92f1976d815144c9b9ff779309ff79ed9dd3 (in the new_features branch, has not been pulled to master).

As a side note, we should probably be running the new_features version of taxomachine on the dev server. I can merge it with master but was waiting because it's possible that some integration might need to touchups. It has all the new features I've been working on and is more consistent and cleaner. But it should be tested... Let me know if you want me to merge it to master.

curl -X POST http://localhost:7474/db/data/ext/OTTServices/graphdb/getTaxonInfo -H "content-type:application/json" -d '{"ottId":822825,"includeLineage":true}'

{
  "ot:ottId" : 822825,
  "rank" : "species",
  "flags" : [ ],
  "taxonomic_lineage" : [ {
    "ot:ottId" : 655485,
    "rank" : "genus",
    "flags" : [ ],
    "ot:ottTaxonName" : "Datura",
    "uniqname" : "",
    "node_id" : 3282971
  }, {
    "ot:ottId" : 484909,
    "rank" : "tribe",
    "flags" : [ ],
    "ot:ottTaxonName" : "Datureae",
    "uniqname" : "",
    "node_id" : 3282970
  }, {
    "ot:ottId" : 13867,
    "rank" : "subfamily",
    "flags" : [ "SIBLING_LOWER" ],
    "ot:ottTaxonName" : "Solanoideae",
    "uniqname" : "",
    "node_id" : 3282882
  }, {
    "ot:ottId" : 473827,
    "rank" : "family",
    "flags" : [ "EDITED", "SIBLING_LOWER" ],
    "ot:ottTaxonName" : "Solanaceae",
    "uniqname" : "",
    "node_id" : 3282292
  }, {
    "ot:ottId" : 1050255,
    "rank" : "order",
    "flags" : [ "EDITED", "SIBLING_LOWER" ],
    "ot:ottTaxonName" : "Solanales",
    "uniqname" : "",
    "node_id" : 3282274
  }, {
    "ot:ottId" : 596112,
    "rank" : "no rank",
    "flags" : [ ],
    "ot:ottTaxonName" : "lamiids",
    "uniqname" : "",
    "node_id" : 3246706
  }, {
    "ot:ottId" : 1008294,
    "rank" : "subclass",
    "flags" : [ "SIBLING_LOWER" ],
    "ot:ottTaxonName" : "asterids",
    "uniqname" : "",
    "node_id" : 3177782
  }, {
    "ot:ottId" : 5316182,
    "rank" : "no rank",
    "flags" : [ ],
    "ot:ottTaxonName" : "Pentapetalae",
    "uniqname" : "",
    "node_id" : 3159192
  }, {
    "ot:ottId" : 853757,
    "rank" : "no rank",
    "flags" : [ ],
    "ot:ottTaxonName" : "Gunneridae",
    "uniqname" : "",
    "node_id" : 3159127
  }, {
    "ot:ottId" : 431495,
    "rank" : "no rank",
    "flags" : [ ],
    "ot:ottTaxonName" : "eudicotyledons",
    "uniqname" : "",
    "node_id" : 3153633
  }, {
    "ot:ottId" : 5298374,
    "rank" : "no rank",
    "flags" : [ ],
    "ot:ottTaxonName" : "Mesangiospermae",
    "uniqname" : "",
    "node_id" : 3143382
  }, {
    "ot:ottId" : 99252,
    "rank" : "phylum",
    "flags" : [ ],
    "ot:ottTaxonName" : "Magnoliophyta",
    "uniqname" : "",
    "node_id" : 3143378
  }, {
    "ot:ottId" : 10218,
    "rank" : "no rank",
    "flags" : [ ],
    "ot:ottTaxonName" : "Spermatophyta",
    "uniqname" : "",
    "node_id" : 3143376
  }, {
    "ot:ottId" : 1007992,
    "rank" : "no rank",
    "flags" : [ ],
    "ot:ottTaxonName" : "Euphyllophyta",
    "uniqname" : "",
    "node_id" : 3143374
  }, {
    "ot:ottId" : 10210,
    "rank" : "no rank",
    "flags" : [ ],
    "ot:ottTaxonName" : "Tracheophyta",
    "uniqname" : "",
    "node_id" : 3142443
  }, {
    "ot:ottId" : 5342313,
    "rank" : "no rank",
    "flags" : [ ],
    "ot:ottTaxonName" : "Embryophyta",
    "uniqname" : "",
    "node_id" : 3142439
  }, {
    "ot:ottId" : 916750,
    "rank" : "phylum",
    "flags" : [ "EDITED", "SIBLING_LOWER" ],
    "ot:ottTaxonName" : "Streptophyta",
    "uniqname" : "",
    "node_id" : 3139100
  }, {
    "ot:ottId" : 361838,
    "rank" : "kingdom",
    "flags" : [ "EDITED", "SIBLING_LOWER" ],
    "ot:ottTaxonName" : "Chloroplastida",
    "uniqname" : "",
    "node_id" : 3139091
  }, {
    "ot:ottId" : 5268475,
    "rank" : "no rank",
    "flags" : [ ],
    "ot:ottTaxonName" : "Archaeplastida",
    "uniqname" : "",
    "node_id" : 3139090
  }, {
    "ot:ottId" : 304358,
    "rank" : "domain",
    "flags" : [ "EDITED" ],
    "ot:ottTaxonName" : "Eukaryota",
    "uniqname" : "",
    "node_id" : 135626
  }, {
    "ot:ottId" : 93302,
    "rank" : "no rank",
    "flags" : [ ],
    "ot:ottTaxonName" : "cellular organisms",
    "uniqname" : "",
    "node_id" : 135624
  }, {
    "ot:ottId" : 805080,
    "rank" : "no rank",
    "flags" : [ ],
    "ot:ottTaxonName" : "life",
    "uniqname" : "",
    "node_id" : 2
  } ],
  "synonyms" : [ "Datura stremonium", "Datura tatula", "Datura stramonium", "jimsonweed", "common thornapple" ],
  "ot:ottTaxonName" : "Datura stramonium",
  "uniqname" : "",
  "node_id" : 3282999
}

[jimallman]

As a side note, we should probably be running the new_features version of taxomachine on the dev server.

No problem, I've updated ot10.config here to use the new_features branch: https://github.com/OpenTreeOfLife/deployed-systems/commit/c2c5252af809d6eff2a795b507879b09903d9d1e

The test of getTaxonInfo above works, though there's no lineage reported from devapi.opentreeoflife.org. Other operations are failing: We can't load the synthetic tree properly, nor can we load studies into the curator app. I'm chasing these now...

[jar398]

Did the new taxomachine plugin get pushed to devapi?

On Thu, Jul 17, 2014 at 12:08 AM, Jim Allman notifications@github.com wrote:

As a side note, we should probably be running the new_features version of taxomachine on the dev server.

No problem, I've updated ot10.config here to use the new_features branch: OpenTreeOfLife/deployed-systems@c2c5252 https://github.com/OpenTreeOfLife/deployed-systems/commit/c2c5252af809d6eff2a795b507879b09903d9d1e

The test of getTaxonInfo above works, though there's no lineage reported from devapi.opentreeoflife.org. Other operations are failing: We can't load the synthetic tree properly, nor can we load studies into the curator app. I'm chasing these now...

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49256460 .

[jimallman]

Did the new taxomachine plugin get pushed to devapi?

@jar398: Yes, but everything's broken at the moment. I'm trying to sort this out.

[chinchliff]

It's going to need a new database to go with it... I'll get the upload process going in a few mins

On Thursday, July 17, 2014, Jim Allman notifications@github.com wrote:

Did the new taxomachine plugin get pushed to devapi?

@jar398 https://github.com/jar398: Yes, but everything's broken at the moment. I'm trying to sort this out.

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49305593 .

[jimallman]

OK, thanks. I thought that might be an issue.

I also ran into a problem in the latest treemachine (maven failed to find stuff in gnu.trove.*).. This is fixed in https://github.com/OpenTreeOfLife/treemachine/commit/7164e539c8e50e24d21a626843e69645efac0361

[chinchliff]

I don't think devapi.opentreeoflife.org has the most recent taxomachine plugin. When I use curl to query the server, it returns information suggesting it's still using an old version. For example, on my local machine, this curl call produces:

curl http://localhost:7474/db/data/ext/OTTServices

{
  "graphdb" : {
    "getTaxonInfo" : {
      "extends" : "graphdb",
      "description" : "Get information about a known taxon. If the option
to include info about the lineage is used, the information will be provided
in an ordered array, with the least inclusive taxa at lower indices (i.e.
higher indices are higher taxa).",
      "name" : "getTaxonInfo",
      "parameters" : [ {
        "description" : "The OTT id of the taxon of interest.",
        "optional" : false,
        "name" : "ottId",
        "type" : "long"
      }, {
        "description" : "Whether or not to include information about all
the higher level taxa that include this one.",
        "optional" : true,
        "name" : "includeLineage",
        "type" : "boolean"
      } ]
    },
    "getDeprecatedTaxa" : {
      "extends" : "graphdb",
      "description" : "Get a list of deprecated ott ids and the names to
which they correspond.",
      "name" : "getDeprecatedTaxa",
      "parameters" : [ ]
    },
    "getTaxonomyInfo" : {
      "extends" : "graphdb",
      "description" : "Get information about the taxonomy itself.",
      "name" : "getTaxonomyInfo",
      "parameters" : [ ]
    }
  }
}

But on devapi, it produces:

curl http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices

{
  "message" : "No such ServerPlugin: \"OTTServices\"",
  "exception" : "PluginLookupException",
  "fullname" : "org.neo4j.server.plugins.PluginLookupException",
  "stacktrace" : [
"org.neo4j.server.plugins.PluginManager.describeAll(PluginManager.java:150)",
"org.neo4j.server.rest.web.ExtensionService.extensionList(ExtensionService.java:293)",
"org.neo4j.server.rest.web.ExtensionService.getExtensionList(ExtensionService.java:107)",
"java.lang.reflect.Method.invoke(Method.java:606)",
"org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"
]
}

And, the call to the getTaxonInfo plugin doesn't have the includeLineage parameter (which I just added yesterday)

curl http://devapi.opentreeoflife.org//txomachine/v1/getTaxonInfo

{
  "extends" : "graphdb",
  "description" : "Get information about a recognized taxon.",
  "name" : "getTaxonInfo",
  "parameters" : [ {
    "description" : "The OTT id of the taxon of interest.",
    "optional" : false,
    "name" : "ottId",
    "type" : "long"
  } ]
}

[jar398]

This could be the dependencies bug we saw before. The thing to do is to delete the .jar file and reinstall. I'll see if I can make the deployment script do this.

On Thu, Jul 17, 2014 at 12:41 PM, Cody Hinchliff notifications@github.com wrote:

I don't think devapi.opentreeoflife.org has the most recent taxomachine plugin. When I use curl to query the server, it returns information suggesting it's still using an old version. For example, on my local machine, this curl call produces:

curl http://localhost:7474/db/data/ext/OTTServices

{
"graphdb" : {
"getTaxonInfo" : {
"extends" : "graphdb",
"description" : "Get information about a known taxon. If the option
to include info about the lineage is used, the information will be provided
in an ordered array, with the least inclusive taxa at lower indices (i.e.
higher indices are higher taxa).",
"name" : "getTaxonInfo",
"parameters" : [ {
"description" : "The OTT id of the taxon of interest.",
"optional" : false,
"name" : "ottId",
"type" : "long"
}, {
"description" : "Whether or not to include information about all
the higher level taxa that include this one.",
"optional" : true,
"name" : "includeLineage",
"type" : "boolean"
} ]
},
"getDeprecatedTaxa" : {
"extends" : "graphdb",
"description" : "Get a list of deprecated ott ids and the names to
which they correspond.",
"name" : "getDeprecatedTaxa",
"parameters" : [ ]
},
"getTaxonomyInfo" : {
"extends" : "graphdb",
"description" : "Get information about the taxonomy itself.",
"name" : "getTaxonomyInfo",
"parameters" : [ ]
}
}
}

But on devapi, it produces:

curl http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices

{
"message" : "No such ServerPlugin: \"OTTServices\"",
"exception" : "PluginLookupException",
"fullname" : "org.neo4j.server.plugins.PluginLookupException",
"stacktrace" : [

"org.neo4j.server.plugins.PluginManager.describeAll(PluginManager.java:150)",

"org.neo4j.server.rest.web.ExtensionService.extensionList(ExtensionService.java:293)",

"org.neo4j.server.rest.web.ExtensionService.getExtensionList(ExtensionService.java:107)",
"java.lang.reflect.Method.invoke(Method.java:606)",

"org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"
]
}

And, the call to the getTaxonInfo plugin doesn't have the includeLineage parameter (which I just added yesterday)

curl http://devapi.opentreeoflife.org//txomachine/v1/getTaxonInfo

{
"extends" : "graphdb",
"description" : "Get information about a recognized taxon.",
"name" : "getTaxonInfo",
"parameters" : [ {
"description" : "The OTT id of the taxon of interest.",
"optional" : false,
"name" : "ottId",
"type" : "long"
} ]
}

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49332272 .

[jar398]

can you try again now? sorry about that. I've added a patch for the dependence problem, not what I wanted to do, but it might work next time.

On Thu, Jul 17, 2014 at 12:42 PM, Cody Hinchliff notifications@github.com wrote:

I don't think devapi.opentreeoflife.org has the most recent taxomachine plugin. When I use curl to query the server, it returns information suggesting it's still using an old version. For example, on my local machine, this curl call produces:

curl http://localhost:7474/db/data/ext/OTTServices

{

"graphdb" : { "getTaxonInfo" : { "extends" : "graphdb", "description" : "Get information about a known taxon. If the optionto include info about the lineage is used, the information will be providedin an ordered array, with the least inclusive taxa at lower indices (i.e.higher indices are higher taxa).", "name" : "getTaxonInfo", "parameters" : [ { "description" : "The OTT id of the taxon of interest.", "optional" : false, "name" : "ottId", "type" : "long" }, { "description" : "Whether or not to include information about allthe higher level taxa that include this one.", "optional" : true, "name" : "includeLineage", "type" : "boolean" } ] }, "getDeprecatedTaxa" : { "extends" : "graphdb", "description" : "Get a list of deprecated ott ids and the names towhich they correspond.", "name" : "getDeprecatedTaxa", "parameters" : [ ] }, "getTaxonomyInfo" : { "extends" : "graphdb", "description" : "Get information about the taxonomy itself.", "name" : "getTaxonomyInfo", "parameters" : [ ] } }}

But on devapi, it produces:

curl http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices

{

"message" : "No such ServerPlugin: \"OTTServices\"", "exception" : "PluginLookupException", "fullname" : "org.neo4j.server.plugins.PluginLookupException", "stacktrace" : ["org.neo4j.server.plugins.PluginManager.describeAll(PluginManager.java:150)","org.neo4j.server.rest.web.ExtensionService.extensionList(ExtensionService.java:293)","org.neo4j.server.rest.web.ExtensionService.getExtensionList(ExtensionService.java:107)","java.lang.reflect.Method.invoke(Method.java:606)","org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"]}

And, the call to the getTaxonInfo plugin doesn't have the includeLineage parameter (which I just added yesterday)

curl http://devapi.opentreeoflife.org//txomachine/v1/getTaxonInfo

{ "extends" : "graphdb", "description" : "Get information about a recognized taxon.", "name" : "getTaxonInfo", "parameters" : [ { "description" : "The OTT id of the taxon of interest.", "optional" : false, "name" : "ottId", "type" : "long" } ]}

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49332383 .

[jar398]

Hmm. Still losing. I believe this is the new_features branch, is that right? The plugin got rebuilt, I looked.

[INFO] Building jar: /home/opentree/repo/taxomachine/target/taxomachine-neo4j-plugins-0.0.1-SNAPSHOT.jar

[INFO]

[INFO] BUILD SUCCESS ... (venv)opentree@ashby:~$ wget http://localhost:7474/db/data/ext/OTTServices --2014-07-17 18:05:00-- http://localhost:7474/db/data/ext/OTTServices Resolving localhost (localhost)... ::1, 127.0.0.1 Connecting to localhost (localhost)|::1|:7474... failed: Connection refused. Connecting to localhost (localhost)|127.0.0.1|:7474... connected. HTTP request sent, awaiting response... 404 Not Found 2014-07-17 18:05:00 ERROR 404: Not Found.

... ls -l neo4j-taxomachine/plugins total 152 -rw-r--r-- 1 opentree opentree 374 Nov 11 2013 README.txt drwxr-sr-x 2 opentree opentree 4096 Jun 10 22:43 gremlin1.5 -rw-r--r-- 1 opentree opentree 143725 Jul 17 18:03 taxomachine-neo4j-plugins-0.0.1-SNAPSHOT.jar ... (venv)opentree@ashby:~$ tail log/messages Thu Jul 17 16:17:49 BST 2014 jima ot10 Install: peyotl setup.py develop run Thu Jul 17 16:18:01 BST 2014 jima ot10 Checkout: taxomachine commit c33143a421aa40f6038a0d524984a505d113d48d Thu Jul 17 16:18:14 BST 2014 jima ot10 Finished installing neo4j instances Thu Jul 17 16:18:18 BST 2014 jima ot10 Checkout: treemachine commit 7164e539c8e50e24d21a626843e69645efac0361 Thu Jul 17 16:18:52 BST 2014 jima ot10 Started treemachine Thu Jul 17 16:18:52 BST 2014 jima ot10 Finished installing neo4j instances Thu Jul 17 16:18:56 BST 2014 jima ot10 Finished installing neo4j instances Thu Jul 17 17:12:17 BST 2014 cody ot10 Installed taxomachine neo4j database Thu Jul 17 18:03:37 BST 2014 jar ot10 Started taxomachine Thu Jul 17 18:03:37 BST 2014 jar ot10 Finished installing neo4j instances (venv)opentree@ashby:~$

On Thu, Jul 17, 2014 at 1:04 PM, Jonathan A Rees rees@mumble.net wrote:

can you try again now? sorry about that. I've added a patch for the dependence problem, not what I wanted to do, but it might work next time.

On Thu, Jul 17, 2014 at 12:42 PM, Cody Hinchliff <notifications@github.com

wrote:

I don't think devapi.opentreeoflife.org has the most recent taxomachine plugin. When I use curl to query the server, it returns information suggesting it's still using an old version. For example, on my local machine, this curl call produces:

curl http://localhost:7474/db/data/ext/OTTServices

{

"graphdb" : { "getTaxonInfo" : { "extends" : "graphdb", "description" : "Get information about a known taxon. If the optionto include info about the lineage is used, the information will be providedin an ordered array, with the least inclusive taxa at lower indices (i.e.higher indices are higher taxa).", "name" : "getTaxonInfo", "parameters" : [ { "description" : "The OTT id of the taxon of interest.", "optional" : false, "name" : "ottId", "type" : "long" }, { "description" : "Whether or not to include information about allthe higher level taxa that include this one.", "optional" : true, "name" : "includeLineage", "type" : "boolean" } ] }, "getDeprecatedTaxa" : { "extends" : "graphdb", "description" : "Get a list of deprecated ott ids and the names towhich they correspond.", "name" : "getDeprecatedTaxa", "parameters" : [ ] }, "getTaxonomyInfo" : { "extends" : "graphdb", "description" : "Get information about the taxonomy itself.", "name" : "getTaxonomyInfo", "parameters" : [ ] } }}

But on devapi, it produces:

curl http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices

{

"message" : "No such ServerPlugin: \"OTTServices\"", "exception" : "PluginLookupException", "fullname" : "org.neo4j.server.plugins.PluginLookupException", "stacktrace" : ["org.neo4j.server.plugins.PluginManager.describeAll(PluginManager.java:150)","org.neo4j.server.rest.web.ExtensionService.extensionList(ExtensionService.java:293)","org.neo4j.server.rest.web.ExtensionService.getExtensionList(ExtensionService.java:107)","java.lang.reflect.Method.invoke(Method.java:606)","org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"]}

And, the call to the getTaxonInfo plugin doesn't have the includeLineage parameter (which I just added yesterday)

curl http://devapi.opentreeoflife.org//txomachine/v1/getTaxonInfo

{ "extends" : "graphdb", "description" : "Get information about a recognized taxon.", "name" : "getTaxonInfo", "parameters" : [ { "description" : "The OTT id of the taxon of interest.", "optional" : false, "name" : "ottId", "type" : "long" } ]}

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49332383 .

[chinchliff]

The plugin seems to be updated now, I see the new parameters and can call the OTTServices methods. But something is broken. It appears to be some goofy maven business again, withe the NoClassDefFound errors for classes that should be there.

curl -X POST http:/ devapi.opentreeoflife.org/taxomachine/ext/OTTServices/graphdb/getTaxonInfo -H "content-type:application/json" -d '{"ottId":822825,"includeLineage":true}' { "message" : "org/opentree/exceptions/MultipleHitsException", "exception" : "NoClassDefFoundError", "fullname" : "java.lang.NoClassDefFoundError", "stacktrace" : [ "org.opentree.taxonomy.plugins.OTTServices.getTaxonInfo(OTTServices.java:73)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.plugins.PluginMethod.invoke(PluginMethod.java:57)", "org.neo4j.server.plugins.PluginManager.invoke(PluginManager.java:168)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:300)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:122)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)" ], "cause" : { "message" : "org.opentree.exceptions.MultipleHitsException", "exception" : "ClassNotFoundException", "stacktrace" : [ "java.net.URLClassLoader$1.run(URLClassLoader.java:366)", "java.net.URLClassLoader$1.run(URLClassLoader.java:355)", "java.security.AccessController.doPrivileged(Native Method)", "java.net.URLClassLoader.findClass(URLClassLoader.java:354)", "java.lang.ClassLoader.loadClass(ClassLoader.java:425)", "sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)", "java.lang.ClassLoader.loadClass(ClassLoader.java:358)", "org.opentree.taxonomy.plugins.OTTServices.getTaxonInfo(OTTServices.java:73)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.plugins.PluginMethod.invoke(PluginMethod.java:57)", "org.neo4j.server.plugins.PluginManager.invoke(PluginManager.java:168)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:300)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:122)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)" ], "fullname" : "java.lang.ClassNotFoundException" } }

On Thu, Jul 17, 2014 at 1:11 PM, Jonathan A Rees notifications@github.com wrote:

Hmm. Still losing. I believe this is the new_features branch, is that right? The plugin got rebuilt, I looked.

[INFO] Building jar: /home/opentree/repo/taxomachine/target/taxomachine-neo4j-plugins-0.0.1-SNAPSHOT.jar

[INFO]

[INFO] BUILD SUCCESS ... (venv)opentree@ashby:~$ wget http://localhost:7474/db/data/ext/OTTServices --2014-07-17 18:05:00-- http://localhost:7474/db/data/ext/OTTServices Resolving localhost (localhost)... ::1, 127.0.0.1 Connecting to localhost (localhost)|::1|:7474... failed: Connection refused. Connecting to localhost (localhost)|127.0.0.1|:7474... connected. HTTP request sent, awaiting response... 404 Not Found 2014-07-17 18:05:00 ERROR 404: Not Found.

... ls -l neo4j-taxomachine/plugins total 152 -rw-r--r-- 1 opentree opentree 374 Nov 11 2013 README.txt drwxr-sr-x 2 opentree opentree 4096 Jun 10 22:43 [0m [01;34mgremlin1.5 [0m -rw-r--r-- 1 opentree opentree 143725 Jul 17 18:03 taxomachine-neo4j-plugins-0.0.1-SNAPSHOT.jar ... (venv)opentree@ashby:~$ tail log/messages Thu Jul 17 16:17:49 BST 2014 jima ot10 Install: peyotl setup.py develop run Thu Jul 17 16:18:01 BST 2014 jima ot10 Checkout: taxomachine commit c33143a421aa40f6038a0d524984a505d113d48d Thu Jul 17 16:18:14 BST 2014 jima ot10 Finished installing neo4j instances Thu Jul 17 16:18:18 BST 2014 jima ot10 Checkout: treemachine commit 7164e539c8e50e24d21a626843e69645efac0361 Thu Jul 17 16:18:52 BST 2014 jima ot10 Started treemachine Thu Jul 17 16:18:52 BST 2014 jima ot10 Finished installing neo4j instances Thu Jul 17 16:18:56 BST 2014 jima ot10 Finished installing neo4j instances Thu Jul 17 17:12:17 BST 2014 cody ot10 Installed taxomachine neo4j database Thu Jul 17 18:03:37 BST 2014 jar ot10 Started taxomachine Thu Jul 17 18:03:37 BST 2014 jar ot10 Finished installing neo4j instances (venv)opentree@ashby:~$

On Thu, Jul 17, 2014 at 1:04 PM, Jonathan A Rees rees@mumble.net wrote:

can you try again now? sorry about that. I've added a patch for the dependence problem, not what I wanted to do, but it might work next time.

On Thu, Jul 17, 2014 at 12:42 PM, Cody Hinchliff < notifications@github.com

wrote:

I don't think devapi.opentreeoflife.org has the most recent taxomachine plugin. When I use curl to query the server, it returns information suggesting it's still using an old version. For example, on my local machine, this curl call produces:

curl http://localhost:7474/db/data/ext/OTTServices

{

"graphdb" : { "getTaxonInfo" : { "extends" : "graphdb", "description" : "Get information about a known taxon. If the optionto include info about the lineage is used, the information will be providedin an ordered array, with the least inclusive taxa at lower indices (i.e.higher indices are higher taxa).", "name" : "getTaxonInfo", "parameters" : [ { "description" : "The OTT id of the taxon of interest.", "optional" : false, "name" : "ottId", "type" : "long" }, { "description" : "Whether or not to include information about allthe higher level taxa that include this one.", "optional" : true, "name" : "includeLineage", "type" : "boolean" } ] }, "getDeprecatedTaxa" : { "extends" : "graphdb", "description" : "Get a list of deprecated ott ids and the names towhich they correspond.", "name" : "getDeprecatedTaxa", "parameters" : [ ] }, "getTaxonomyInfo" : { "extends" : "graphdb", "description" : "Get information about the taxonomy itself.", "name" : "getTaxonomyInfo", "parameters" : [ ] } }}

But on devapi, it produces:

curl http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices

{

"message" : "No such ServerPlugin: \"OTTServices\"", "exception" : "PluginLookupException", "fullname" : "org.neo4j.server.plugins.PluginLookupException", "stacktrace" : ["org.neo4j.server.plugins.PluginManager.describeAll(PluginManager.java:150)","org.neo4j.server.rest.web.ExtensionService.extensionList(ExtensionService.java:293)","org.neo4j.server.rest.web.ExtensionService.getExtensionList(ExtensionService.java:107)","java.lang.reflect.Method.invoke(Method.java:606)","org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"]}

And, the call to the getTaxonInfo plugin doesn't have the includeLineage parameter (which I just added yesterday)

curl http://devapi.opentreeoflife.org//txomachine/v1/getTaxonInfo

{ "extends" : "graphdb", "description" : "Get information about a recognized taxon.", "name" : "getTaxonInfo", "parameters" : [ { "description" : "The OTT id of the taxon of interest.", "optional" : false, "name" : "ottId", "type" : "long" } ]}

— Reply to this email directly or view it on GitHub < https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49332383>

.

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49335698 .

[chinchliff]

In the past we have usually been able to fix this by just recompiling.

On Thu, Jul 17, 2014 at 1:13 PM, Cody Hinchliff cody.hinchliff@gmail.com wrote:

The plugin seems to be updated now, I see the new parameters and can call the OTTServices methods. But something is broken. It appears to be some goofy maven business again, withe the NoClassDefFound errors for classes that should be there.

curl -X POST http:/ devapi.opentreeoflife.org/taxomachine/ext/OTTServices/graphdb/getTaxonInfo -H "content-type:application/json" -d '{"ottId":822825,"includeLineage":true}' { "message" : "org/opentree/exceptions/MultipleHitsException", "exception" : "NoClassDefFoundError", "fullname" : "java.lang.NoClassDefFoundError", "stacktrace" : [ "org.opentree.taxonomy.plugins.OTTServices.getTaxonInfo(OTTServices.java:73)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.plugins.PluginMethod.invoke(PluginMethod.java:57)", "org.neo4j.server.plugins.PluginManager.invoke(PluginManager.java:168)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:300)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:122)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)" ], "cause" : { "message" : "org.opentree.exceptions.MultipleHitsException", "exception" : "ClassNotFoundException", "stacktrace" : [ "java.net.URLClassLoader$1.run(URLClassLoader.java:366)", "java.net.URLClassLoader$1.run(URLClassLoader.java:355)", "java.security.AccessController.doPrivileged(Native Method)", "java.net.URLClassLoader.findClass(URLClassLoader.java:354)", "java.lang.ClassLoader.loadClass(ClassLoader.java:425)", "sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)", "java.lang.ClassLoader.loadClass(ClassLoader.java:358)", "org.opentree.taxonomy.plugins.OTTServices.getTaxonInfo(OTTServices.java:73)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.plugins.PluginMethod.invoke(PluginMethod.java:57)", "org.neo4j.server.plugins.PluginManager.invoke(PluginManager.java:168)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:300)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:122)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)" ], "fullname" : "java.lang.ClassNotFoundException" } }

On Thu, Jul 17, 2014 at 1:11 PM, Jonathan A Rees <notifications@github.com

wrote:

Hmm. Still losing. I believe this is the new_features branch, is that right? The plugin got rebuilt, I looked.

[INFO] Building jar: /home/opentree/repo/taxomachine/target/taxomachine-neo4j-plugins-0.0.1-SNAPSHOT.jar

[INFO]

[INFO] BUILD SUCCESS ... (venv)opentree@ashby:~$ wget http://localhost:7474/db/data/ext/OTTServices --2014-07-17 18:05:00-- http://localhost:7474/db/data/ext/OTTServices Resolving localhost (localhost)... ::1, 127.0.0.1 Connecting to localhost (localhost)|::1|:7474... failed: Connection refused. Connecting to localhost (localhost)|127.0.0.1|:7474... connected. HTTP request sent, awaiting response... 404 Not Found 2014-07-17 18:05:00 ERROR 404: Not Found.

... ls -l neo4j-taxomachine/plugins total 152 -rw-r--r-- 1 opentree opentree 374 Nov 11 2013 README.txt drwxr-sr-x 2 opentree opentree 4096 Jun 10 22:43 [0m [01;34mgremlin1.5 [0m -rw-r--r-- 1 opentree opentree 143725 Jul 17 18:03 taxomachine-neo4j-plugins-0.0.1-SNAPSHOT.jar ... (venv)opentree@ashby:~$ tail log/messages Thu Jul 17 16:17:49 BST 2014 jima ot10 Install: peyotl setup.py develop run Thu Jul 17 16:18:01 BST 2014 jima ot10 Checkout: taxomachine commit c33143a421aa40f6038a0d524984a505d113d48d Thu Jul 17 16:18:14 BST 2014 jima ot10 Finished installing neo4j instances Thu Jul 17 16:18:18 BST 2014 jima ot10 Checkout: treemachine commit 7164e539c8e50e24d21a626843e69645efac0361 Thu Jul 17 16:18:52 BST 2014 jima ot10 Started treemachine Thu Jul 17 16:18:52 BST 2014 jima ot10 Finished installing neo4j instances Thu Jul 17 16:18:56 BST 2014 jima ot10 Finished installing neo4j instances Thu Jul 17 17:12:17 BST 2014 cody ot10 Installed taxomachine neo4j database Thu Jul 17 18:03:37 BST 2014 jar ot10 Started taxomachine Thu Jul 17 18:03:37 BST 2014 jar ot10 Finished installing neo4j instances (venv)opentree@ashby:~$

On Thu, Jul 17, 2014 at 1:04 PM, Jonathan A Rees rees@mumble.net wrote:

can you try again now? sorry about that. I've added a patch for the dependence problem, not what I wanted to do, but it might work next time.

On Thu, Jul 17, 2014 at 12:42 PM, Cody Hinchliff < notifications@github.com

wrote:

I don't think devapi.opentreeoflife.org has the most recent taxomachine plugin. When I use curl to query the server, it returns information suggesting it's still using an old version. For example, on my local machine, this curl call produces:

curl http://localhost:7474/db/data/ext/OTTServices

{

"graphdb" : { "getTaxonInfo" : { "extends" : "graphdb", "description" : "Get information about a known taxon. If the optionto include info about the lineage is used, the information will be providedin an ordered array, with the least inclusive taxa at lower indices (i.e.higher indices are higher taxa).", "name" : "getTaxonInfo", "parameters" : [ { "description" : "The OTT id of the taxon of interest.", "optional" : false, "name" : "ottId", "type" : "long" }, { "description" : "Whether or not to include information about allthe higher level taxa that include this one.", "optional" : true, "name" : "includeLineage", "type" : "boolean" } ] }, "getDeprecatedTaxa" : { "extends" : "graphdb", "description" : "Get a list of deprecated ott ids and the names towhich they correspond.", "name" : "getDeprecatedTaxa", "parameters" : [ ] }, "getTaxonomyInfo" : { "extends" : "graphdb", "description" : "Get information about the taxonomy itself.", "name" : "getTaxonomyInfo", "parameters" : [ ] } }}

But on devapi, it produces:

curl http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices

{

"message" : "No such ServerPlugin: \"OTTServices\"", "exception" : "PluginLookupException", "fullname" : "org.neo4j.server.plugins.PluginLookupException", "stacktrace" : ["org.neo4j.server.plugins.PluginManager.describeAll(PluginManager.java:150)","org.neo4j.server.rest.web.ExtensionService.extensionList(ExtensionService.java:293)","org.neo4j.server.rest.web.ExtensionService.getExtensionList(ExtensionService.java:107)","java.lang.reflect.Method.invoke(Method.java:606)","org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"]}

And, the call to the getTaxonInfo plugin doesn't have the includeLineage parameter (which I just added yesterday)

curl http://devapi.opentreeoflife.org//txomachine/v1/getTaxonInfo

{ "extends" : "graphdb", "description" : "Get information about a recognized taxon.", "name" : "getTaxonInfo", "parameters" : [ { "description" : "The OTT id of the taxon of interest.", "optional" : false, "name" : "ottId", "type" : "long" } ]}

— Reply to this email directly or view it on GitHub < https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49332383>

.

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49335698 .

[jar398]

I did recompile; do you mean we have to blow away the maven jar cache?

On Thu, Jul 17, 2014 at 1:15 PM, Cody Hinchliff notifications@github.com wrote:

In the past we have usually been able to fix this by just recompiling.

On Thu, Jul 17, 2014 at 1:13 PM, Cody Hinchliff cody.hinchliff@gmail.com

wrote:

The plugin seems to be updated now, I see the new parameters and can call the OTTServices methods. But something is broken. It appears to be some goofy maven business again, withe the NoClassDefFound errors for classes that should be there.

curl -X POST http:/

devapi.opentreeoflife.org/taxomachine/ext/OTTServices/graphdb/getTaxonInfo -H "content-type:application/json" -d '{"ottId":822825,"includeLineage":true}' { "message" : "org/opentree/exceptions/MultipleHitsException", "exception" : "NoClassDefFoundError", "fullname" : "java.lang.NoClassDefFoundError", "stacktrace" : [

"org.opentree.taxonomy.plugins.OTTServices.getTaxonInfo(OTTServices.java:73)",

"java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.plugins.PluginMethod.invoke(PluginMethod.java:57)", "org.neo4j.server.plugins.PluginManager.invoke(PluginManager.java:168)",

"org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:300)",

"org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:122)",

"java.lang.reflect.Method.invoke(Method.java:606)",

"org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"

], "cause" : { "message" : "org.opentree.exceptions.MultipleHitsException", "exception" : "ClassNotFoundException", "stacktrace" : [ "java.net.URLClassLoader$1.run(URLClassLoader.java:366)", "java.net.URLClassLoader$1.run(URLClassLoader.java:355)", "java.security.AccessController.doPrivileged(Native Method)", "java.net.URLClassLoader.findClass(URLClassLoader.java:354)", "java.lang.ClassLoader.loadClass(ClassLoader.java:425)", "sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)", "java.lang.ClassLoader.loadClass(ClassLoader.java:358)",

"org.opentree.taxonomy.plugins.OTTServices.getTaxonInfo(OTTServices.java:73)",

"java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.plugins.PluginMethod.invoke(PluginMethod.java:57)", "org.neo4j.server.plugins.PluginManager.invoke(PluginManager.java:168)",

"org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:300)",

"org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:122)",

"java.lang.reflect.Method.invoke(Method.java:606)",

"org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"

], "fullname" : "java.lang.ClassNotFoundException" } }

On Thu, Jul 17, 2014 at 1:11 PM, Jonathan A Rees < notifications@github.com

wrote:

Hmm. Still losing. I believe this is the new_features branch, is that right? The plugin got rebuilt, I looked.

[INFO] Building jar:

/home/opentree/repo/taxomachine/target/taxomachine-neo4j-plugins-0.0.1-SNAPSHOT.jar

[INFO]

---

[INFO] BUILD SUCCESS ... (venv)opentree@ashby:~$ wget http://localhost:7474/db/data/ext/OTTServices --2014-07-17 18:05:00-- http://localhost:7474/db/data/ext/OTTServices Resolving localhost (localhost)... ::1, 127.0.0.1 Connecting to localhost (localhost)|::1|:7474... failed: Connection refused. Connecting to localhost (localhost)|127.0.0.1|:7474... connected. HTTP request sent, awaiting response... 404 Not Found 2014-07-17 18:05:00 ERROR 404: Not Found.

... ls -l neo4j-taxomachine/plugins total 152 -rw-r--r-- 1 opentree opentree 374 Nov 11 2013 README.txt drwxr-sr-x 2 opentree opentree 4096 Jun 10 22:43 [0m [01;34mgremlin1.5 [0m -rw-r--r-- 1 opentree opentree 143725 Jul 17 18:03 taxomachine-neo4j-plugins-0.0.1-SNAPSHOT.jar ... (venv)opentree@ashby:~$ tail log/messages Thu Jul 17 16:17:49 BST 2014 jima ot10 Install: peyotl setup.py develop run Thu Jul 17 16:18:01 BST 2014 jima ot10 Checkout: taxomachine commit c33143a421aa40f6038a0d524984a505d113d48d Thu Jul 17 16:18:14 BST 2014 jima ot10 Finished installing neo4j instances Thu Jul 17 16:18:18 BST 2014 jima ot10 Checkout: treemachine commit 7164e539c8e50e24d21a626843e69645efac0361 Thu Jul 17 16:18:52 BST 2014 jima ot10 Started treemachine Thu Jul 17 16:18:52 BST 2014 jima ot10 Finished installing neo4j instances Thu Jul 17 16:18:56 BST 2014 jima ot10 Finished installing neo4j instances Thu Jul 17 17:12:17 BST 2014 cody ot10 Installed taxomachine neo4j database Thu Jul 17 18:03:37 BST 2014 jar ot10 Started taxomachine Thu Jul 17 18:03:37 BST 2014 jar ot10 Finished installing neo4j instances (venv)opentree@ashby:~$

On Thu, Jul 17, 2014 at 1:04 PM, Jonathan A Rees rees@mumble.net wrote:

can you try again now? sorry about that. I've added a patch for the dependence problem, not what I wanted to do, but it might work next time.

On Thu, Jul 17, 2014 at 12:42 PM, Cody Hinchliff < notifications@github.com

wrote:

I don't think devapi.opentreeoflife.org has the most recent taxomachine plugin. When I use curl to query the server, it returns information suggesting it's still using an old version. For example, on my local machine, this curl call produces:

curl http://localhost:7474/db/data/ext/OTTServices

{

"graphdb" : { "getTaxonInfo" : { "extends" : "graphdb", "description" : "Get information about a known taxon. If the optionto include info about the lineage is used, the information will be providedin an ordered array, with the least inclusive taxa at lower indices (i.e.higher indices are higher taxa).", "name" : "getTaxonInfo", "parameters" : [ { "description" : "The OTT id of the taxon of interest.", "optional" : false, "name" : "ottId", "type" : "long" }, { "description" : "Whether or not to include information about allthe higher level taxa that include this one.", "optional" : true, "name" : "includeLineage", "type" : "boolean" } ] }, "getDeprecatedTaxa" : { "extends" : "graphdb", "description" : "Get a list of deprecated ott ids and the names towhich they correspond.", "name" : "getDeprecatedTaxa", "parameters" : [ ] }, "getTaxonomyInfo" : { "extends" : "graphdb", "description" : "Get information about the taxonomy itself.", "name" : "getTaxonomyInfo", "parameters" : [ ] } }}

But on devapi, it produces:

curl http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices

{

"message" : "No such ServerPlugin: \"OTTServices\"", "exception" : "PluginLookupException", "fullname" : "org.neo4j.server.plugins.PluginLookupException", "stacktrace" :

["org.neo4j.server.plugins.PluginManager.describeAll(PluginManager.java:150)","org.neo4j.server.rest.web.ExtensionService.extensionList(ExtensionService.java:293)","org.neo4j.server.rest.web.ExtensionService.getExtensionList(ExtensionService.java:107)","java.lang.reflect.Method.invoke(Method.java:606)","org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"]}

And, the call to the getTaxonInfo plugin doesn't have the includeLineage parameter (which I just added yesterday)

curl http://devapi.opentreeoflife.org//txomachine/v1/getTaxonInfo

{ "extends" : "graphdb", "description" : "Get information about a recognized taxon.", "name" : "getTaxonInfo", "parameters" : [ { "description" : "The OTT id of the taxon of interest.", "optional" : false, "name" : "ottId", "type" : "long" } ]}

— Reply to this email directly or view it on GitHub <

https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49332383>

.

— Reply to this email directly or view it on GitHub < https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49335698>

.

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49336167 .

[chinchliff]

No, just re-recompile. Sometimes maven compiles seem to produce jars with missing classes. Generally rebuilding the jar (again) fixes the problem.

[jimallman]

I've pushed Cody's oti/compatibility_update branch to ot10 (devapi), but I'm seeing the same errors when testing getTaxonInfo.

For what it's worth, I saw a series of "overlapping classes" messages in the taxomachine build output, then:

[WARNING] maven-shade-plugin has detected that some .class files
[WARNING] are present in two or more JARs. When this happens, only
[WARNING] one single version of the class is copied in the uberjar.
[WARNING] Usually this is not harmful and you can skeep these
[WARNING] warnings, otherwise try to manually exclude artifacts
[WARNING] based on mvn dependency:tree -Ddetail=true and the above
[WARNING] output
[WARNING] See http://docs.codehaus.org/display/MAVENUSER/Shade+Plugin

Is it possible that we have conflicting classes (different versions) for different services? @chinchliff , if you're seeing these messages in your local build, it's probably a red herring.

[chinchliff]

Hmm. Maybe try wiping out the ~/.m2/ directory that holds all the maven-installed jars and recompiling. If maven was pointing to old versions of the ot-base jars that could indeed produce those missing class def errors.

On Thu, Jul 17, 2014 at 4:11 PM, Jim Allman notifications@github.com wrote:

I've pushed Cody's oti/compatibility_update branch to ot10 (devapi), but I'm seeing the same errors when testing getTaxonInfo.

For what it's worth, I saw a series of "overlapping classes" messages in the taxomachine build output, then:

[WARNING] maven-shade-plugin has detected that some .class files [WARNING] are present in two or more JARs. When this happens, only [WARNING] one single version of the class is copied in the uberjar. [WARNING] Usually this is not harmful and you can skeep these [WARNING] warnings, otherwise try to manually exclude artifacts [WARNING] based on mvn dependency:tree -Ddetail=true and the above [WARNING] output [WARNING] See http://docs.codehaus.org/display/MAVENUSER/Shade+Plugin

Is it possible that we have conflicting classes (different versions) for different services? @chinchliff https://github.com/chinchliff , if you're seeing these messages in your local build, it's probably a red herring.

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49358976 .

[jimallman]

@chinchliff : I'm quoting my own email here for the record: It looks like the exception is thrown here: https://github.com/OpenTreeOfLife/taxomachine/blob/93f369c9e949bbdc2623429182dead2aaabfddea/src/main/java/org/opentree/taxonomy/Taxonomy.java#L147

I see that we’re looking for OTVocabularyPredicate.OT_OTT_ID.propertyName(), which is imported from here:

import org.opentree.properties.OTVocabularyPredicate;

...but I don’t see where these properties are defined. Should there be a ‘properties.java’ file here? https://github.com/OpenTreeOfLife/taxomachine/tree/new_features/src/main/java/org/opentree

Or is this introduced in some other way? I don’t see any definition of OT_OTT_ID, just code that’s using it: https://github.com/OpenTreeOfLife/taxomachine/search?q=OT_OTT_ID&type=Code

Could it be in another repo, perhaps? or an uncommitted file on your local machine?

[chinchliff]

It's in the ot-base jar, which is installed into the local maven repo, and accessed by oti, taxomachine, and treemachine.

Hm. This is confusing. I don't think there is a problem with OTVocabularyPredicate.OT_OTT_ID.propertyName(). The no class def error seems to be associated with the MultipleHitsException class. That exception should also be defined in the ot-base jar. I don't think it's an uncommitted file issue, I haven't touched that class file in months and things are working on all my machines.

The ".properties." part indicates that it's in that subpackage, which corresponds to a directory in the repo/jar. So in this case, in the ot-base jar, there should be a file /org/opentree/exceptions/MultipleHitsException.class. And that file should be getting copied into the oti plugin jar by maven post-compilation. But it sounds like maven's getting confused and that class isn't making it into the jarfile. It could be some other problem, but that's my best guess. Definitely worth wiping out the maven cache at ~/.m2/ and trying it again (maven will rebuild its cache). This often solves weird maven compilation glitches for me, though it is obviously annoying and I would love to know a better way, I just don't know of one.

On Thu, Jul 17, 2014 at 4:44 PM, Jim Allman notifications@github.com wrote:

@chinchliff https://github.com/chinchliff : I'm quoting my own email here for the record: It looks like the exception is thrown here:

https://github.com/OpenTreeOfLife/taxomachine/blob/93f369c9e949bbdc2623429182dead2aaabfddea/src/main/java/org/opentree/taxonomy/Taxonomy.java#L147

I see that we’re looking for OTVocabularyPredicate.OT_OTT_ID.propertyName(), which is imported from here:

import org.opentree.properties.OTVocabularyPredicate;

...but I don’t see where these properties are defined. Should there be a ‘properties.java’ file here?

https://github.com/OpenTreeOfLife/taxomachine/tree/new_features/src/main/java/org/opentree

Or is this introduced in some other way? I don’t see any definition of OT_OTT_ID, just code that’s using it: https://github.com/OpenTreeOfLife/taxomachine/search?q=OT_OTT_ID&type=Code

Could it be in another repo, perhaps? or an uncommitted file on your local machine?

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49362915 .

[jimallman]

Definitely worth wiping out the maven cache at ~/.m2/ and trying it again (maven will rebuild its cache).

Sadly, in this case I clobbered ~/.m2/ and re-deployed everything on ot11 (devapi), and the result is the same:

curl -X POST http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices/graphdb/getTaxonInfo -H "content-type:application/json" -d '{"ottId":822825,"includeLineage":true}'
{
  "message" : "org/opentree/exceptions/MultipleHitsException",
  "exception" : "NoClassDefFoundError",
  ...

[jimallman]

@chinchliff , when I run /home/opentree/repo/taxomachine/mvn_serverplugins.sh as admin on ot10, I see this error:

Downloading: http://repo.maven.apache.org/maven2/commons-beanutils/commons-beanutils/1.8.0/commons-beanutils-1.8.0.pom
Downloaded: http://repo.maven.apache.org/maven2/commons-beanutils/commons-beanutils/1.8.0/commons-beanutils-1.8.0.pom (11 KB at 329.2 KB/sec)
[WARNING] The POM for org.opentree:ot-base:jar:0.0.1-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.opentree:jade:jar:0.0.1-SNAPSHOT is missing, no dependency information available

Running the same script as user opentree works fine, and the resulting .jar file seems to contain the expected classes with OT_OTT_ID:

$ cd ~/repo/taxomachine
$ ./mvn_serverplugins.sh

Lots of chatter, and a new jar is created successfully. I replaced the active plugins jar with this and restarted neo4j:

$ cd ~/neo4j-taxomachine/plugins
$ mv taxomachine-neo4j-plugins-0.0.1-SNAPSHOT.jar taxomachine-neo4j-plugins-0.0.1-SNAPSHOT.jar.OLD
$ cp ~/repo/taxomachine/target/taxomachine-neo4j-plugins-0.0.1-SNAPSHOT.jar .
$ ~/neo4j-taxomachine/bin/neo4j restart

Now I get a refreshing new error when I try to trace taxon IDs:

 curl -X POST http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices/graphdb/getTaxonInfo -H "content-type:application/json" -d '{"ottId":822825,"includeLineage":true}'
{
  "message" : "'ot:ottTaxonName' property not found for NodeImpl#3282999.",
  "exception" : "NotFoundException",
  "fullname" : "org.neo4j.graphdb.NotFoundException",
  "stacktrace" : [ "org.neo4j.kernel.impl.core.Primitive.newPropertyNotFoundException(Primitive.java:193)", "org.neo4j.kernel.impl.core.Primitive.getProperty(Primitive.java:188)", "org.neo4j.kernel.impl.core.NodeImpl.getProperty(NodeImpl.java:53)", "org.neo4j.kernel.impl.core.NodeProxy.getProperty(NodeProxy.java:155)", "org.opentree.taxonomy.plugins.OTTServices.addPropertyFromNode(OTTServices.java:135)", "org.opentree.taxonomy.plugins.OTTServices.addTaxonInfo(OTTServices.java:120)", "org.opentree.taxonomy.plugins.OTTServices.getTaxonInfo(OTTServices.java:91)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.plugins.PluginMethod.invoke(PluginMethod.java:57)", "org.neo4j.server.plugins.PluginManager.invoke(PluginManager.java:168)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:300)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:122)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)" ]
}

$ curl -X POST http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices/graphdb/getTaxonInfo -H "content-type:application/json" -d '{"ottId":822825,"includeLineage":true}'
curl -X POST http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices/graphdb/getTaxonInfo -H "content-type:application/json" -d '{"ottId":81461,"includeLineage":true}'
{
  "message" : "'ot:ottTaxonName' property not found for NodeImpl#678634.",
  "exception" : "NotFoundException",
  "fullname" : "org.neo4j.graphdb.NotFoundException",
  "stacktrace" : [ "org.neo4j.kernel.impl.core.Primitive.newPropertyNotFoundException(Primitive.java:193)", "org.neo4j.kernel.impl.core.Primitive.getProperty(Primitive.java:188)", "org.neo4j.kernel.impl.core.NodeImpl.getProperty(NodeImpl.java:53)", "org.neo4j.kernel.impl.core.NodeProxy.getProperty(NodeProxy.java:155)", "org.opentree.taxonomy.plugins.OTTServices.addPropertyFromNode(OTTServices.java:135)", "org.opentree.taxonomy.plugins.OTTServices.addTaxonInfo(OTTServices.java:120)", "org.opentree.taxonomy.plugins.OTTServices.getTaxonInfo(OTTServices.java:91)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.plugins.PluginMethod.invoke(PluginMethod.java:57)", "org.neo4j.server.plugins.PluginManager.invoke(PluginManager.java:168)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:300)", "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:122)", "java.lang.reflect.Method.invoke(Method.java:606)", "org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)" ]
}

Is this progress? I can't tell anymore..

[jimallman]

@chinchliff, I found another possible clue in this discussion of "POM ... missing, no dependency information available" messages. Here's another similar thread on the same subject.

A common topic in both is the importance of the .m2/settings.xml file and especially its <repository> elements.

Could this explain how you get different results on your local system versus ot10?

[chinchliff]

How refreshing!

Perhaps the maven settings are different. I never modified any of my maven settings though...

When you clobbered the ~/.m2/ dir, which user did you do that under? It looks like under admin the errors are due to not having installed the ot-base and jade repos, which require manual installs (because they are not on the maven server).

That error under the opentree user about the property not being found is a good sign. It suggests the new code is being used, but the taxomachine database is still in the old format, though oddly I thought I uploaded a new one... Anyway I have an even newer one with ott2.8draft5 that I'll start uploading now, which hopefully should solve that problem.

I'll look into the maven settings file...

On Friday, July 18, 2014, Jim Allman notifications@github.com wrote:

@chinchliff https://github.com/chinchliff, I found another possible clue in this discussion of "POM ... missing, no dependency information available" messages. http://stackoverflow.com/questions/4650460/maven-could-not-resolve-dependencies-problem Here's another similar thread http://cocoon.10839.n7.nabble.com/mvn-clean-install-failure-quot-Could-not-resolve-dependencies-quot-td11041.html on the same subject.

A common topic in both is the importance of the .m2/settings.xml file and especially its elements.

Could this explain how you get different results on your local system versus ot10?

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49399285 .

[jimallman]

When you clobbered the ~/.m2/ dir, which user did you do that under?

To be clear, I removed both /home/admin/.m2 and /home/opentree/.m2.

It looks like under admin the errors are due to not having installed the ot-base and jade repos, which require manual installs (because they are not on the maven server).

Understood. I thought it might be a clue. It looks like this part of the deployment (deploy/setup/install-neo4j-app.sh) always runs as user opentree, and it uses the very same ./mvn_serverplugins.sh script that I used manually. One results in an incomplete jar file, the other works. Odd.

[jimallman]

That error under the opentree user about the property not being found is a good sign. It suggests the new code is being used, but the taxomachine database is still in the old format

Ah, that makes sense. Let me know if you need a second pair of eyes.

[jimallman]

As an aid to troubleshooting, I've cooked up a little script called jargrep.sh that will scan all local .jar files for a given string. It works like this:

$ cd ~/.m2/repository/org/opentree/taxomachine
$ ./jargrep.sh MultipleHitsException
Looking for "MultipleHitsException" in local .jar files...
./0.0.1-SNAPSHOT/taxomachine-0.0.1-SNAPSHOT.jar:
   org/opentree/taxonomy/MainRunner.class
   org/opentree/taxonomy/Taxonomy.class
   org/opentree/tnrs/MultipleHitsException.class
   org/opentree/tnrs/TNRSResults.class

Is there a sensible place for things like this in our repos? It's pretty tiny, so for now I'll paste it here:

#!/bin/bash
if [ "x$1" = x ]; then
    echo Lists all instances of a search string by .jar file and class.
    echo Usage: $0  MyTextToFind
    echo Usage: $0  \"multi-word text to find\"
    exit
fi
# This simple jar-search tool was adapted from
#   http://www.linuxquestions.org/questions/linux-newbie-8/searching-contents-of-class-file-within-jar-for-strings-705087/#post3445838
#
# It's a bit more powerful than 'jar tvf filename.jar', and useful for
# troubleshooting missing classfiles.
echo Looking for \"$1\" in local .jar files...
find . -iname '*.jar' -print | while read jar; do
  echo "$jar:"
  unzip -qq -l $jar | sed 's/.* //' | while read cls; do
    unzip -c $jar $cls | grep -q "$1" && echo "   "$cls
  done
done

[chinchliff]

Him, did you mean ot10? I don't see an ot11 in the deployed-servers repo...

On Friday, July 18, 2014, Jim Allman notifications@github.com wrote:

Definitely worth wiping out the maven cache at ~/.m2/ and trying it again (maven will rebuild its cache).

Sadly, in this case I clobbered ~/.m2/ and re-deployed everything on ot11 (devapi), and the result is the same:

curl -X POST http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices/graphdb/getTaxonInfo -H "content-type:application/json" -d '{"ottId":822825,"includeLineage":true}'{ "message" : "org/opentree/exceptions/MultipleHitsException", "exception" : "NoClassDefFoundError", ...

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49394546 .

[jimallman]

Him, did you mean ot10?

Yes, sorry about that. (ot11 was the old devtree, now on ot16)

[jimallman]

@chinchliff, please pull the latest deployed-systems. I just committed the change in oti from master branch to compatibility_update. (I've been deploying this since yesterday, but forgot to commit the change.)

[chinchliff]

I see it in the opentree-servers.txt file, but I don't see a config file in the development/ dir. I'm wondering if there's a new technique for specifying servers in the push.sh script that I'm not aware of. Are we still using the -c argument to specify a config file?

On Friday, July 18, 2014, Jim Allman notifications@github.com wrote:

@chinchliff https://github.com/chinchliff, please pull the latest deployed-systems. I just committed the change in oti from master branch to compatibility_update. (I've been deploying this since yesterday, but forgot to commit the change.)

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49438052 .

[jimallman]

Yes, it's a bit confusing. We keep track of active (and past) machines in opentree-servers.txt, but most of the real action is in the server-specific config files. In this case, the file development/ot10.config is where we define things like the branch of each repo to deploy.

[chinchliff]

Right. But I don't see a config file for ot11. So I am a bit confused, since you had mentioned ot11 in your earlier email. Which config file should I be using?

On Friday, July 18, 2014, Jim Allman notifications@github.com wrote:

Yes, it's a bit confusing. We keep track of active (and past) machines in opentree-servers.txt https://github.com/OpenTreeOfLife/deployed-systems/blob/master/opentree-servers.txt, but most of the real action is in the server-specific config files. In this case, the file development/ot10.config https://github.com/OpenTreeOfLife/deployed-systems/blob/17b1ef802f1932ed9f57b67fdece535822150d92/development/ot10.config#L26 is where we define things like the branch of each repo to deploy.

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49450152 .

[jimallman]

Sorry for the confusion. Note that any blue text in the messages above is a link.

I had written ot11 earlier when I meant ot10, as explained above. ot11 has been retired, so now the development system uses ot16 (for webapps) and ot10 (for APIs).

The 'compatibility_update' branch is specified for oti in ot10.config (click here to see the exact line).

[chinchliff]

Ok. Alles ist klar. Pushing the new taxomachine db to ot10 now.

On Friday, July 18, 2014, Jim Allman notifications@github.com wrote:

Sorry for the confusion. Note that any blue text in the messages above is a link.

I had written ot11 earlier when I meant ot10, as explained above https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49437790. ot11 has been retired, so now the development system uses ot16 (for webapps) and ot10 (for APIs).

The 'compatibility_update' branch is specified for oti in ot10.config (click here to see the exact line) https://github.com/OpenTreeOfLife/deployed-systems/blob/17b1ef802f1932ed9f57b67fdece535822150d92/development/ot10.config#L26 .

— Reply to this email directly or view it on GitHub https://github.com/OpenTreeOfLife/taxomachine/issues/50#issuecomment-49454567 .

[chinchliff]

This now seems to be working on the dev server. Closing this issue (reopen if necessary).

curl -X POST http://devapi.opentreeoflife.org/taxomachine/ext/OTTServices/graphdb/getTaxonInfo -H "content-type:application/json" -d '{"ottId":822825,"includeLineage":true}'