phylotastic / phylotastic-portal

Web tool (ruby on rails) providing access to phylotastic services for acquiring species trees.
https://github.com/phylotastic/phylotastic/wiki
1 stars 3 forks source link

popularity service no list returned #319

Open arlin opened 6 years ago

arlin commented 6 years ago

2 cases were due to the popularity service, but Anura (frogs) eems to be our problem. This may be because this name occurs twice in OT taxonomy. Here is the pop service call: http://beta.onezoom.org/popularity/list?key=0&otts=991547&expand_taxa=True&max=20&names=True&sort=rank

ducvan0212 commented 6 years ago

The popularity service return no species for your provided taxa

http://phylo.cs.nmsu.edu:5006/phylotastic_ws/ts/popular_species?taxon=Anura&num_species=20
{"status_code": 200, "message": "Success", "popular_species": [], "input_taxon": "Anura", "meta_data": {"execution_time": 0.79, "creation_time": "2018-06-26T18:02:51.678243", "source_urls": ["http://beta.onezoom.org"]}}
arlin commented 6 years ago

OK, but the original OneZoom service returns 20 species (http://beta.onezoom.org/popularity/list?key=0&otts=991547&expand_taxa=True&max=20&names=True&sort=rank). See below. I guess @abusalehmdtayeen wrapped this and we need to ask him?

  "max_taxa_out": 100,
  "n_taxa": 6555,
  "header": [
    "ott",
    "popularity",
    "popularity_rank",
    "name"
  ],
  "tot_spp": 2123183,
  "max_taxa_in": 5,
  "data": [
    [
      558313,
      120126.871261261,
      4969,
      "Bombina pachypus"
    ],
    [
      485827,
      119544.16658687402,
      5232,
      "Ascaphus truei"
    ],
    [
      809935,
      119544.16658687402,
      5232,
      "Ascaphus montanus"
    ],
    [
      95656,
      119533.46173383704,
      5236,
      "Barbourula kalimantanensis"
    ],
    [
      95643,
      119523.8761594891,
      5239,
      "Barbourula busuangensis"
    ],
    [
      3617970,
      119079.38859691223,
      5440,
      "Latonia nigriventer"
    ],
    [
      459012,
      118578.68808144129,
      5667,
      "Rhinophrynus dorsalis"
    ],
    [
      3617971,
      118566.16104987256,
      5671,
      "Alytes obstretricans"
    ],
    [
      991557,
      118425.81345044229,
      5720,
      "Bombina orientalis"
    ],
    [
      142397,
      118421.89158551843,
      5721,
      "Leiopelma hochstetteri"
    ],
    [
      3617968,
      118421.89158551843,
      5721,
      "Leiopelma markhami"
    ],
    [
      140874,
      118388.24284026785,
      5726,
      "Pseudhymenochirus merlini"
    ],
    [
      698730,
      118377.93666731074,
      5728,
      "Discoglossus scovazzi"
    ],
    [
      1039096,
      118140.43563374496,
      5852,
      "Bombina maxima"
    ],
    [
      3617953,
      118136.41959641801,
      5856,
      "Hymenochirus boulengeri"
    ],
    [
      315705,
      117808.87040968648,
      6026,
      "Hadromophryne natalensis"
    ],
    [
      385874,
      117778.88075466585,
      6039,
      "Alytes cisternasii"
    ],
    [
      3617967,
      117604.51628590077,
      6099,
      "Leiopelma auroraensis"
    ],
    [
      991559,
      117545.05699190534,
      6122,
      "Bombina bombina"
    ],
    [
      248872,
      117490.94951563881,
      6150,
      "Leiopelma archeyi"
    ]
  ]
}
abusalehmdtayeen commented 6 years ago

The problem happens when we get multiple matches for a taxon name from OpenTreeofLife API. By default, I pick the first match with score 1.0, get its ott_id and then use the One Zoom API. Since OneZoom API directly works on the ott_id and you are giving the correct ott_id you get results. Below is the TNRS match result for "Anura" from OpenTreeofLife. json { "governing_code" : "undefined", "unambiguous_name_ids" : [ "Anura" ], "unmatched_name_ids" : [ ], "matched_name_ids" : [ "Anura" ], "context" : "All life", "includes_deprecated_taxa" : false, "includes_dubious_names" : false, "includes_approximate_matches" : false, "taxonomy" : { "weburl" : "https://tree.opentreeoflife.org/about/taxonomy-version/ott3.0", "author" : "open tree of life project", "name" : "ott", "source" : "ott3.0draft6", "version" : "3.0" }, "results" : [ { "id" : "Anura", "matches" : [ { "matched_node_id" : 852780, "synonyms" : [ "Anura" ], "flags" : [ "BARREN", "SIBLING_HIGHER" ], "ot:ottTaxonName" : "Anura", "search_string" : "anura", "matched_name" : "Anura", "is_synonym" : false, "score" : 1.0, "tax_sources" : [ "gbif:3089470" ], "unique_name" : "Anura (genus in kingdom Archaeplastida)", "ot:ottId" : 4728082, "is_deprecated" : false, "nomenclature_code" : "ICN", "is_approximate_match" : false, "rank" : "genus", "is_dubious" : false }, { "matched_node_id" : 4695959, "synonyms" : [ "Anura", "Vieraellidae", "Comobatrachus aenigmatis", "Eobatrachus agilis", "Rana cordofana", "Salientia" ], "flags" : [ ], "ot:ottTaxonName" : "Anura", "search_string" : "anura", "matched_name" : "Anura", "is_synonym" : false, "score" : 1.0, "tax_sources" : [ "ncbi:8342", "worms:448306", "gbif:952", "irmng:10211" ], "unique_name" : "Anura (order in Opisthokonta)", "ot:ottId" : 991547, "is_deprecated" : false, "nomenclature_code" : "ICZN", "is_approximate_match" : false, "rank" : "order", "is_dubious" : false }, { "matched_node_id" : 850148, "synonyms" : [ "Schmalhausenia", "Anura", "Cousinia", "Tiarocarpus", "Hypacanthium", "Lipskyella" ], "flags" : [ ], "ot:ottTaxonName" : "Cousinia", "search_string" : "anura", "matched_name" : "Anura", "is_synonym" : true, "score" : 1.0, "tax_sources" : [ "ncbi:92908", "gbif:3101721", "irmng:1304742" ], "unique_name" : "Cousinia", "ot:ottId" : 597720, "is_deprecated" : false, "nomenclature_code" : "ICN", "is_approximate_match" : false, "rank" : "genus", "is_dubious" : false }, { "matched_node_id" : 2137502, "synonyms" : [ "Kaiparanura", "Laminilabrum", "Nawenia", "Pisanianura", "Anura" ], "flags" : [ ], "ot:ottTaxonName" : "Pisanianura", "search_string" : "anura", "matched_name" : "Anura", "is_synonym" : true, "score" : 1.0, "tax_sources" : [ "worms:138353", "gbif:2301718", "irmng:1019139" ], "unique_name" : "Pisanianura", "ot:ottId" : 2916683, "is_deprecated" : false, "nomenclature_code" : "ICZN", "is_approximate_match" : false, "rank" : "genus", "is_dubious" : false }, { "matched_node_id" : 4662636, "synonyms" : [ "Anura", "Glossonycteris", "Lonchoglossa", "Anoura" ], "flags" : [ ], "ot:ottTaxonName" : "Anoura", "search_string" : "anura", "matched_name" : "Anura", "is_synonym" : true, "score" : 1.0, "tax_sources" : [ "ncbi:27641", "gbif:5963037", "irmng:1299161" ], "unique_name" : "Anoura", "ot:ottId" : 351791, "is_deprecated" : false, "nomenclature_code" : "ICZN", "is_approximate_match" : false, "rank" : "genus", "is_dubious" : false }, { "matched_node_id" : 4136398, "synonyms" : [ "Anoura", "Neanura", "Anura", "Achoreutes", "Biloba", "Achorutes" ], "flags" : [ ], "ot:ottTaxonName" : "Neanura", "search_string" : "anura", "matched_name" : "Anura", "is_synonym" : true, "score" : 1.0, "tax_sources" : [ "ncbi:106919", "gbif:2122073", "irmng:1016699" ], "unique_name" : "Neanura", "ot:ottId" : 99848, "is_deprecated" : false, "nomenclature_code" : "ICZN", "is_approximate_match" : false, "rank" : "genus", "is_dubious" : false } ] } ] }

abusalehmdtayeen commented 6 years ago

To resolve this, we may need to ask the user to provide another input. For example, if the user mentions the rank as "order" in case of "Anura", you will get the correct result.

arlin commented 6 years ago

But if the user wants to make a tree for frogs, then this is not a solution. There is no other taxon that maps to frogs.

At the very least, we need to tell the user what is happening, e.g., "The taxon could not be mapped unambiguously (see [search ressult)[https://tree.opentreeoflife.org/taxonomy/browse?name=anura]).

Another possibility is to do what OT does on One way to do this would be to do what OT does on pages like this and offer a box that says "enter name or id". This way, the user who wants to resolve an ambiguity can do so by specifying the id instead of the name.

The taxonomy search interface in the main OT viewer provides a pull-down in case of ambiguous matches:

screen shot 2018-06-26 at 8 54 50 pm
ducvan0212 commented 6 years ago

Which is the api in OT that returns different taxonomies in different kingdoms as Arlin showed?

abusalehmdtayeen commented 6 years ago

Hi @ducvan0212 , You don't need a separate service now. I have modified the popularity service to include popular species of all matches of the input taxon. So, you can show the user different results based on their choice. Here is an example output. {'status_code': 200, 'message': 'Success', 'meta_data': {'execution_time': 4.09, 'creation_time': '2018-06-28T07:37:26.712801', 'source_urls': ['http://beta.onezoom.org']}, 'result': [{'popular_species': [{'score': 120126.871261261, 'name': u'Bombina pachypus', 'rank': 4969, 'ott_id': 558313}, {'score': 119544.16658687402, 'name': u'Ascaphus truei', 'rank': 5232, 'ott_id': 485827}, {'score': 119544.16658687402, 'name': u'Ascaphus montanus', 'rank': 5232, 'ott_id': 809935}, {'score': 119533.46173383704, 'name': u'Barbourula kalimantanensis', 'rank': 5236, 'ott_id': 95656}, {'score': 119523.8761594891, 'name': u'Barbourula busuangensis', 'rank': 5239, 'ott_id': 95643}, {'score': 119079.38859691223, 'name': u'Latonia nigriventer', 'rank': 5440, 'ott_id': 3617970}, {'score': 118578.68808144129, 'name': u'Rhinophrynus dorsalis', 'rank': 5667, 'ott_id': 459012}, {'score': 118566.16104987256, 'name': u'Alytes obstretricans', 'rank': 5671, 'ott_id': 3617971}, {'score': 118425.81345044229, 'name': u'Bombina orientalis', 'rank': 5720, 'ott_id': 991557}, {'score': 118421.89158551843, 'name': u'Leiopelma hochstetteri', 'rank': 5721, 'ott_id': 142397}], 'matched_taxon': u'Anura (order in Opisthokonta)', 'ott_id': 991547}, {'popular_species': [{'score': 40754.49695426256, 'name': u'Cousinia platylepis', 'rank': 1450303, 'ott_id': 167185}, {'score': 40754.49695426256, 'name': u'Cousinia turcomanica', 'rank': 1450303, 'ott_id': 193795}, {'score': 40591.795731046244, 'name': u'Cousinia longifolia', 'rank': 1454424, 'ott_id': 100314}, {'score': 40591.795731046244, 'name': u'Cousinia scariosa', 'rank': 1454424, 'ott_id': 244924}, {'score': 40591.795731046244, 'name': u'Cousinia tenella', 'rank': 1454424, 'ott_id': 741656}, {'score': 40512.04880314099, 'name': u'Cousinia triceps', 'rank': 1456371, 'ott_id': 193797}, {'score': 40512.04880314099, 'name': u'Cousinia rufidula', 'rank': 1456371, 'ott_id': 824923}, {'score': 40512.04880314099, 'name': u'Cousinia ruaseola', 'rank': 1456371, 'ott_id': 6057204}, {'score': 40433.33863077187, 'name': u'Cousinia badghysi', 'rank': 1458493, 'ott_id': 519036}, {'score': 40433.33863077187, 'name': u'Cousinia pannosa', 'rank': 1458493, 'ott_id': 870243}], 'matched_taxon': u'Cousinia', 'ott_id': 597720}, {'popular_species': [{'score': 127458.40045718732, 'name': u'Anoura geoffroyi', 'rank': 1704, 'ott_id': 688667}, {'score': 126994.34046910117, 'name': u'Anoura caudifer', 'rank': 1838, 'ott_id': 351792}, {'score': 126264.81032663805, 'name': u'Anoura latidens', 'rank': 2110, 'ott_id': 130223}, {'score': 126223.51628932233, 'name': u'Anoura cultrata', 'rank': 2130, 'ott_id': 485774}], 'matched_taxon': u'Anoura', 'ott_id': 351791}, {'popular_species': [{'score': 60354.346699178095, 'name': u'Neanura ambigua', 'rank': 1113002, 'ott_id': 5010759}, {'score': 60166.7057372178, 'name': u'Neanura coronifera', 'rank': 1117340, 'ott_id': 5010767}, {'score': 59982.69032973983, 'name': u'Neanura growae', 'rank': 1121515, 'ott_id': 5010746}, {'score': 59982.69032973983, 'name': u'Neanura ili', 'rank': 1121515, 'ott_id': 5010747}, {'score': 59982.69032973983, 'name': u'Neanura giselae', 'rank': 1121515, 'ott_id': 5010748}, {'score': 59982.69032973983, 'name': u'Neanura bara', 'rank': 1121515, 'ott_id': 5010750}, {'score': 59982.69032973983, 'name': u'Neanura reticulata', 'rank': 1121515, 'ott_id': 6192982}, {'score': 59802.18072019912, 'name': u'Neanura palmeri', 'rank': 1125869, 'ott_id': 5010745}, {'score': 59802.18072019912, 'name': u'Neanura barberi', 'rank': 1125869, 'ott_id': 5010753}, {'score': 59802.18072019912, 'name': u'Neanura aleuta', 'rank': 1125869, 'ott_id': 5010760}], 'matched_taxon': u'Neanura', 'ott_id': 99848}], 'input_taxon': 'Anura'}