figure out tree retrieval issue

[arlin]

From the load testing results here:

https://docs.google.com/document/d/1mgal-qCKOlGGrHC0EsEve2KzxP_LCy_q2Y2_ytOeM6c/edit#heading=h.fzugdk5hu7q2

figure out how to reproduce the tree retrieval issue.

[abusalehmdtayeen]

There were mainly two tree retrieval issues found in the doc. The first one is the subset of Dima's list that Dr. Pontelli used. This failure was reproduced using the list. The problem was a Unicode error thrown by the ETE component used in the service. The service was able to get the tree from the OpenTreeofLife, but it uses the ETE component to get the number of tips of the tree to show in the result. One of the taxons (Pilosella ×auriculoides) in the tree had a unknown character that caused the encoding issue. This has been fixed by using proper encoding. Now the list gives a tree with 108 tips. The input list is attached.

The second one is the 10 names list of Mustelidae family used by Chas. The exact list was not available, so the error could not be reproduced. But when tested with the wiki link of Mustelidae (https://en.wikipedia.org/wiki/Mustelidae) as input the tree retrieval was successful. random_513_names.txt

[abusalehmdtayeen]

In addition, 1000 names from Dima's list was tested. The portal was able to display a tree with 188 tips. The input list is attached for retesting. random_1000_names.txt

[ChasParr]

Here is the generated Mustelidae list I used. list.txt

[arlin]

I'm getting an error from this list. @abusalehmdtayeen could you investigate please?

[abusalehmdtayeen]

@arlin, I have checked Chas's list. I get the following error from OpenTreeofLife for this list of species.

{
    "message": "Not enough valid node ids provided to construct a subtree (there must be at least two).",
    "exception": "BadInputException",
    "fullname": "org.neo4j.server.rest.repr.BadInputException",
    "stacktrace": [
        "opentree.plugins.tree_of_life_v3.doInducedSubtree(tree_of_life_v3.java:498)",
        "opentree.plugins.tree_of_life.induced_subtree(tree_of_life.java:317)",
        "java.lang.reflect.Method.invoke(Method.java:498)",
        "org.neo4j.server.plugins.PluginMethod.invoke(PluginMethod.java:57)",
        "org.neo4j.server.plugins.PluginManager.invoke(PluginManager.java:168)",
        "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:300)",
        "org.neo4j.server.rest.web.ExtensionService.invokeGraphDatabaseExtension(ExtensionService.java:122)",
        "java.lang.reflect.Method.invoke(Method.java:498)",
        "org.neo4j.server.rest.security.SecurityFilter.doFilter(SecurityFilter.java:112)"
    ]
}

This is a common error that I have also found in some other examples. When @ducvan0212 will start showing the error messages from the service in the portal, it will be clear that sometimes error occurs in the source service for some input cases instead of the wrapper service.

[arlin]

This doesn't completely diagnose the problem for me. I'm pretty sure what is happening is that none of the original names are in the synth tree. If I add Canis lupus and Ursus arctos to the list of 10, then I get a tree with just these two species.

[arlin]

@ChasParr, where did you get that list of mustelids? Are they all extinct? I'm kinda surprised that none of the 10 are in OpenTree's synth tree.

[arlin]

got them from taxon sampling pathway. got 10 names. apparently they are all extinct. we should look into this.

[arlin]

OK, this is closed. We know why the error happened. It is going to spawn a couple of other issues however.