osmose-model / osmose-web-api

Web service that generates Osmose configuration files from data sources like Fishbase and SeaLifeBase. Used by https://www.config.osmose-model.org .
MIT License
2 stars 2 forks source link

Making sure to provide as few NAs as possible for focal functional groups #115

Closed agruss2 closed 7 years ago

agruss2 commented 7 years ago

@jhpoelen This issue is somehow related to #114 For all of the following parameters - It is not reasonable to specify a default value for all the fish and invertebrates recorded in FishBase and SeaLifeBase: (1) predation.accessibility.stage.threshold.sp; (2) predation.predPrey.stage.threshold.sp; (3) species.K.sp; (4) species.length2weight.allometric.power.sp; (5) species.length2weight.condition.factor.sp; (6) species.lifespan.sp; (7) species.lInf.sp; (8) species.maturity.size.sp; (9) species.maturity.age.sp; (10) species.relativefecundity.sp; (11) species.t0.sp*.

Therefore, the default values specified for the 11 abovementioned parameters in the “fishbase-mapping.csv” file should not be used by the API.

Here is what I would like you to implement when estimates are not available for all the species of a given focal functional group to derive a value for one of the 11 abovementioned parameters: (1) Implementing a rule of thumb in the API such that the API uses information for the other species belonging to the genuses, families and orders to which the species of the focal functional group belong. (These other species can be found anywhere across the globe). For the parameter of interest, the API would apply the “data richness” rules we discussed with Skit in #59 to produce an estimate for the focal functional group under consideration. (2) If the API does not manage to obtain an estimate for the parameter of interest after having considered all the species belonging to the genuses, families and orders to which the species of the focal functional group belong -> Setting the value of the parameter of interest to “NA” (not available).

Sorry, this issue is a little bit hard to formulate, so please let me know if anything here is unclear. Many thanks for everything.

jhpoelen commented 7 years ago

If I understand it correctly, you'd like to (a) extend the trait finding algorithm and (b) introduce a way to explicitly state when values are not available using "NA" (or some other "magic" value to identify a value that is unavailable).

Item (b) can be introduced independent of (a).

About (a) - currently, ordered speccodes (species ids) are passed from the wizard to the api. The api then retrieves relevant fishbase trait archives and attempts to retrieve desired traits. If I understand it correctly, you'd like to extend the ordered list (by decreasing richness) for a functional group to include all speccode that shares a genus, family or order with the species included in the functional group. This extended list of speccodes can then be used to retrieve a reasonable parameter value for a specific functional group.

Let's say that a functional group contains 20 species. Each of these species have about 5 sibling species in genus, 20 in family, and 200 in the order. Assuming that these counts are reasonable, and no taxononic overlap between species exist, the list of species would turn to 20 * (5

Before I continue to thinking about ways to implement this, please confirm that we are on the same page.

jhpoelen commented 7 years ago

Note that currently, the api attempts to retrieve values associated to speccodes provided by UI in order. So, if the UI appends some more speccodes beyond the species explicitly mentioned in the functional group, the API should work as usual.

agruss2 commented 7 years ago

@jhpoelen Here is what I think we could do. Let’s say we are dealing with a functional group called “smallpelagics” and the parameter “relative fecundity”. (1) For that “smallpelagics” functional group, the UI has identified, say, 20 species. (2) Before passing information to the API, the UI appends to these 20 species: all the species that belong to the genuses, families and orders of the 20 species. Let’s say that this results in a list of 500 species in total (20 species + 480 appended species). (3) When the API receives the list of 500 species from the UI, it first considers only the 20 species that belong to the “smallpelagics” functional group. -> (i) Case 1: data are available to derive a value for the parameter “relative fecundity” for some of the 20 species that belong to the “smallpelagics” functional group. In that case, the “data richness” rule described in #59 is applied to derive a parameter estimate for the parameter “relative fecundity” for the “smallpelagics” functional group. -> (ii) Case 2: data are not available to derive a value for the parameter “relative fecundity” for none of the 20 species that belong to the “smallpelagics” functional group. In that case, the API will consider the 480 appended species to see if it can derive a value for the parameter “relative fecundity”. (4) If we have faced Case 2 above, then: -> Case 1: data are available to derive a value for the parameter “relative fecundity” for some of the 480 appended species. In that case, the “data richness” rule described in #59 is applied to the list of 480 appended species to derive a parameter estimate for the parameter “relative fecundity” for the “smallpelagics” functional group. -> (ii) Case 2: data are not available to derive a value for the parameter “relative fecundity” for none of the 480 appended species. In that case, the API will provide the value NA for the parameter “relative fecundity” for the “smallpelagics” functional group.

Does this sound good to you? If so, could you please interact with @FIN-casey and @FIN-JBarile to implement what is described above?

jhpoelen commented 7 years ago

@agruss2 Sounds good. As far as I am concerned, no changes are needed in the API - the API gets an ordered list of species for the specific groups and will work down the list until a specific parameter has been found.

@FIN-JBarile @FIN-casey suggest to extend the list of species in a functional group to include the species as described above by @agruss2 . Please let me know if you have any thoughts on this.

agruss2 commented 7 years ago

@FIN-casey @FIN-JBarile Please implement what is described above; and let me know when you think this issue has been solved so that I can run a new test with the bridge between FishBase/SeaLifeBase and OSMOSE. Many thanks.

FIN-casey commented 7 years ago

Hi, @agruss2 , I would like to clarify some things. Shouldn't we have a maximum number of species for the extended list for every functional group? And how do we separate the list of species from the actual list and the extended list? I am planning to add the extended list in the json data before passing it to the API. The nodes of the functional group would be "name","type","taxa", and "othertaxa". The "othertaxa" will be the extended species list listing the name and the url of each of the species just like the format of the "taxa" node. @jhpoelen, The above implementation requires some modification of the API. Please tell me if you think of a better implementation.

agruss2 commented 7 years ago

@FIN-casey Here are my answers to your comments: (1) I think that it would be better not to have a maximum number of species for the extended list for every functional group; this way, we would maximize the probability to obtain a value for parameters. However, I guess that, for computional and practical reasons, you may want me to decide of a maximum number of species for the extended list for every functional group. Is this the case? If so, then please let me know and I'll provide you with a maximum number. (2) You are right that we should think of a way to distinguish between the original list of species and the list of additional species. I think that distinguishing between "taxa" and "othertaxa" is an excellent idea. This way, the API will be able to work first with "taxa" only; and will then consider "othertaxa", if need be.

@jhpoelen Please see my comments above. Could you please make sure that the "functional_groups.csv" file that is provided in the "osmose_config.zip" file lists only "taxa" (and, therefore, does not provide any information about "othertaxa").

jhpoelen commented 7 years ago

@FIN-casey Rather than introducing othertaxa I'd like to suggest to introduce an attribute (or property) in the taxon object.

Currently, we have:

{
  ...
  "groups": [
    {
      "name": "Amberjacks",
      "type": "focal",
      "taxa": [
        {
          "name": "Seriola dumerili",
          "url": "http://fishbase.org/summary/1005"
        },
        {
          "name": "Seriola fasciata",
          "url": "http://fishbase.org/summary/1006"
        }
      ]
    }
    ... 

For taxa related to the explicitly mentioned species in the functional groups, I'd label the taxon object with an optional property like "selectionCriteria" and value "implicit", "inferred" or "same genus". The (omitted) default value for "selectionCriteria" would be "exact".

So, including a related species by genus like Seriola hippos (same genus, not exact match), would produce something like:

{
  ...
  "groups": [
    {
      "name": "Amberjacks",
      "type": "focal",
      "taxa": [
        {
          "name": "Seriola dumerili",
          "url": "http://fishbase.org/summary/1005"
        },
        {
          "name": "Seriola fasciata",
          "url": "http://fishbase.org/summary/1006"
        },
        {
          "name": "Seriola hippos",
          "url": "http://fishbase.org/summary/1960",
          "selectionCriteria": "implicit"
        }
      ]
    }
    ... 

This solution would make it easy to filter or otherwise transform/group taxa by their selection criteria and would require minimal changes to existing code.

jhpoelen commented 7 years ago

@FIN-casey The order of the taxa in the group definition would determine the order in which they are considered. So, if you'd like to make sure to consider the implicitly selected taxa, then add them after the explicitly selected taxa.

Please confirm that this works for you.

jhpoelen commented 7 years ago

On creating an osmose configuration for Iceland Shelf/Sea, I noticed that none of the extended taxa seem to make it into the request from the UI to the API (see below). I've attached the resulting osmose config here: osmose_config (6).zip

@FIN-casey is there any thing the API needs to do to get access to the extended taxon list? If this is not yet implemented, I'd strongly suggest to use the method I suggested in https://github.com/jhpoelen/fb-osmose-bridge/issues/115#issuecomment-311122588 .

Please let me know if there's anything I missed.

This is what got sent from UI to API:

{
  "timeStepsPerYear": "12",
  "groups": [
    {
      "name": "fishpelagicsm",
      "type": "focal",
      "taxa": [
        {
          "name": "Mallotus villosus",
          "url": "http://fishbase.org/summary/252"
        },
        {
          "name": "Benthosema glaciale",
          "url": "http://fishbase.org/summary/21"
        },
        {
          "name": "Notoscopelus kroyeri",
          "url": "http://fishbase.org/summary/27753"
        }
      ]
    },
    {
      "name": "fishpelagicmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Scomber scombrus",
          "url": "http://fishbase.org/summary/118"
        },
        {
          "name": "Sardina pilchardus",
          "url": "http://fishbase.org/summary/1350"
        },
        {
          "name": "Alosa fallax",
          "url": "http://fishbase.org/summary/5355"
        },
        {
          "name": "Trachurus trachurus",
          "url": "http://fishbase.org/summary/1365"
        },
        {
          "name": "Scomberesox saurus",
          "url": "http://fishbase.org/summary/1084"
        },
        {
          "name": "Schedophilus medusophagus",
          "url": "http://fishbase.org/summary/1721"
        }
      ]
    },
    {
      "name": "fishpelagiclg",
      "type": "focal",
      "taxa": [
        {
          "name": "Salmo trutta",
          "url": "http://fishbase.org/summary/238"
        },
        {
          "name": "Sebastes norvegicus",
          "url": "http://fishbase.org/summary/501"
        },
        {
          "name": "Belone belone",
          "url": "http://fishbase.org/summary/47"
        },
        {
          "name": "Brama brama",
          "url": "http://fishbase.org/summary/391"
        },
        {
          "name": "Mola mola",
          "url": "http://fishbase.org/summary/1732"
        },
        {
          "name": "Regalecus glesne",
          "url": "http://fishbase.org/summary/3267"
        }
      ]
    },
    {
      "name": "fishdemersalsm",
      "type": "focal",
      "taxa": [
        {
          "name": "Agonus cataphractus",
          "url": "http://fishbase.org/summary/36"
        },
        {
          "name": "Ammodytes tobianus",
          "url": "http://fishbase.org/summary/38"
        },
        {
          "name": "Liparis liparis",
          "url": "http://fishbase.org/summary/1378"
        },
        {
          "name": "Callionymus maculatus",
          "url": "http://fishbase.org/summary/1807"
        },
        {
          "name": "Ciliata mustela",
          "url": "http://fishbase.org/summary/1290"
        },
        {
          "name": "Ciliata septentrionalis",
          "url": "http://fishbase.org/summary/1873"
        },
        {
          "name": "Triglops murrayi",
          "url": "http://fishbase.org/summary/4146"
        },
        {
          "name": "Liparis montagui",
          "url": "http://fishbase.org/summary/1379"
        },
        {
          "name": "Micrenophrys lilljeborgii",
          "url": "http://fishbase.org/summary/9198"
        }
      ]
    },
    {
      "name": "fishdemersalmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Eutrigla gurnardus",
          "url": "http://fishbase.org/summary/68"
        },
        {
          "name": "Callionymus lyra",
          "url": "http://fishbase.org/summary/23"
        },
        {
          "name": "Myoxocephalus scorpius",
          "url": "http://fishbase.org/summary/1329"
        },
        {
          "name": "Pholis gunnellus",
          "url": "http://fishbase.org/summary/3804"
        },
        {
          "name": "Lumpenus lampretaeformis",
          "url": "http://fishbase.org/summary/1380"
        },
        {
          "name": "Boreogadus saida",
          "url": "http://fishbase.org/summary/319"
        },
        {
          "name": "Enchelyopus cimbrius",
          "url": "http://fishbase.org/summary/1874"
        },
        {
          "name": "Hyperoplus lanceolatus",
          "url": "http://fishbase.org/summary/1374"
        },
        {
          "name": "Sebastes viviparus",
          "url": "http://fishbase.org/summary/1352"
        },
        {
          "name": "Entelurus aequoreus",
          "url": "http://fishbase.org/summary/67"
        },
        {
          "name": "Myxine glutinosa",
          "url": "http://fishbase.org/summary/2513"
        }
      ]
    },
    {
      "name": "fishdemersallg",
      "type": "focal",
      "taxa": [
        {
          "name": "Dicentrarchus labrax",
          "url": "http://fishbase.org/summary/63"
        },
        {
          "name": "Melanogrammus aeglefinus",
          "url": "http://fishbase.org/summary/1381"
        },
        {
          "name": "Pollachius virens",
          "url": "http://fishbase.org/summary/1343"
        },
        {
          "name": "Chelon labrosus",
          "url": "http://fishbase.org/summary/2499"
        },
        {
          "name": "Merluccius merluccius",
          "url": "http://fishbase.org/summary/30"
        },
        {
          "name": "Molva molva",
          "url": "http://fishbase.org/summary/33"
        },
        {
          "name": "Brosme brosme",
          "url": "http://fishbase.org/summary/51"
        },
        {
          "name": "Polyprion americanus",
          "url": "http://fishbase.org/summary/349"
        },
        {
          "name": "Petromyzon marinus",
          "url": "http://fishbase.org/summary/2530"
        },
        {
          "name": "Molva dypterygia",
          "url": "http://fishbase.org/summary/1383"
        },
        {
          "name": "Anarhichas lupus",
          "url": "http://fishbase.org/summary/2501"
        },
        {
          "name": "Conger conger",
          "url": "http://fishbase.org/summary/301"
        },
        {
          "name": "Anarhichas minor",
          "url": "http://fishbase.org/summary/3811"
        }
      ]
    },
    {
      "name": "fishbathypelagicsm",
      "type": "focal",
      "taxa": [
        {
          "name": "Myctophum punctatum",
          "url": "http://fishbase.org/summary/1328"
        },
        {
          "name": "Maurolicus muelleri",
          "url": "http://fishbase.org/summary/754"
        },
        {
          "name": "Argyropelecus hemigymnus",
          "url": "http://fishbase.org/summary/6968"
        },
        {
          "name": "Argyropelecus olfersii",
          "url": "http://fishbase.org/summary/5097"
        }
      ]
    },
    {
      "name": "fishbathypelagicmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Hoplostethus atlanticus",
          "url": "http://fishbase.org/summary/334"
        },
        {
          "name": "Micromesistius poutassou",
          "url": "http://fishbase.org/summary/31"
        },
        {
          "name": "Sebastes mentella",
          "url": "http://fishbase.org/summary/505"
        },
        {
          "name": "Coryphaenoides mediterraneus",
          "url": "http://fishbase.org/summary/8175"
        },
        {
          "name": "Coryphaenoides brevibarbis",
          "url": "http://fishbase.org/summary/27726"
        }
      ]
    },
    {
      "name": "fishbathypelagiclg",
      "type": "focal",
      "taxa": [
        {
          "name": "Aphanopus carbo",
          "url": "http://fishbase.org/summary/646"
        },
        {
          "name": "Coryphaenoides rupestris",
          "url": "http://fishbase.org/summary/332"
        },
        {
          "name": "Lampris guttatus",
          "url": "http://fishbase.org/summary/1072"
        },
        {
          "name": "Trachipterus arcticus",
          "url": "http://fishbase.org/summary/3265"
        },
        {
          "name": "Centrolophus niger",
          "url": "http://fishbase.org/summary/249"
        },
        {
          "name": "Nemichthys scolopaceus",
          "url": "http://fishbase.org/summary/2660"
        },
        {
          "name": "Coryphaenoides armatus",
          "url": "http://fishbase.org/summary/5127"
        }
      ]
    },
    {
      "name": "fishbathydemersalmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Argentina silus",
          "url": "http://fishbase.org/summary/2700"
        },
        {
          "name": "Argentina sphyraena",
          "url": "http://fishbase.org/summary/20"
        },
        {
          "name": "Coryphaenoides guentheri",
          "url": "http://fishbase.org/summary/8176"
        },
        {
          "name": "Echiodon drummondii",
          "url": "http://fishbase.org/summary/65"
        },
        {
          "name": "Careproctus reinhardti",
          "url": "http://fishbase.org/summary/15500"
        },
        {
          "name": "Bathysaurus ferox",
          "url": "http://fishbase.org/summary/11841"
        }
      ]
    },
    {
      "name": "fishbathydemersallg",
      "type": "focal",
      "taxa": [
        {
          "name": "Lophius piscatorius",
          "url": "http://fishbase.org/summary/716"
        },
        {
          "name": "Alepocephalus bairdii",
          "url": "http://fishbase.org/summary/230"
        },
        {
          "name": "Beryx decadactylus",
          "url": "http://fishbase.org/summary/1319"
        },
        {
          "name": "Chimaera monstrosa",
          "url": "http://fishbase.org/summary/2503"
        },
        {
          "name": "Synaphobranchus kaupii",
          "url": "http://fishbase.org/summary/9104"
        }
      ]
    },
    {
      "name": "fishbenthopelagicsm",
      "type": "focal",
      "taxa": [
        {
          "name": "Gasterosteus aculeatus",
          "url": "http://fishbase.org/summary/2420"
        },
        {
          "name": "Ammodytes marinus",
          "url": "http://fishbase.org/summary/37"
        }
      ]
    },
    {
      "name": "fishbenthopelagicmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Clupea harengus",
          "url": "http://fishbase.org/summary/24"
        },
        {
          "name": "Merlangius merlangus",
          "url": "http://fishbase.org/summary/29"
        },
        {
          "name": "Pagellus bogaraveo",
          "url": "http://fishbase.org/summary/890"
        },
        {
          "name": "Trisopterus esmarkii",
          "url": "http://fishbase.org/summary/1023"
        },
        {
          "name": "Cyclopterus lumpus",
          "url": "http://fishbase.org/summary/62"
        },
        {
          "name": "Coelorinchus caelorhincus",
          "url": "http://fishbase.org/summary/1726"
        }
      ]
    },
    {
      "name": "fishbenthopelagiclg",
      "type": "focal",
      "taxa": [
        {
          "name": "Gadus morhua",
          "url": "http://fishbase.org/summary/69"
        },
        {
          "name": "Salmo salar",
          "url": "http://fishbase.org/summary/236"
        },
        {
          "name": "Salvelinus alpinus",
          "url": "http://sealifebase.org/summary/alpinus"
        },
        {
          "name": "Macrourus berglax",
          "url": "http://fishbase.org/summary/331"
        },
        {
          "name": "Pollachius pollachius",
          "url": "http://fishbase.org/summary/34"
        },
        {
          "name": "Phycis blennoides",
          "url": "http://fishbase.org/summary/1340"
        },
        {
          "name": "Anarhichas denticulatus",
          "url": "http://fishbase.org/summary/3810"
        },
        {
          "name": "Argyrosomus regius",
          "url": "http://fishbase.org/summary/418"
        },
        {
          "name": "Notacanthus chemnitzii",
          "url": "http://fishbase.org/summary/2661"
        }
      ]
    },
    {
      "name": "fishreefassociatedsm",
      "type": "focal",
      "taxa": [
        {
          "name": "Lebetus scorpioides",
          "url": "http://fishbase.org/summary/456"
        }
      ]
    },
    {
      "name": "fishsharksmmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Etmopterus princeps",
          "url": "http://fishbase.org/summary/683"
        },
        {
          "name": "Etmopterus spinax",
          "url": "http://fishbase.org/summary/687"
        }
      ]
    },
    {
      "name": "fishsharklg",
      "type": "focal",
      "taxa": [
        {
          "name": "Lamna nasus",
          "url": "http://fishbase.org/summary/88"
        },
        {
          "name": "Squalus acanthias",
          "url": "http://fishbase.org/summary/139"
        },
        {
          "name": "Galeorhinus galeus",
          "url": "http://fishbase.org/summary/4642"
        },
        {
          "name": "Centrophorus squamosus",
          "url": "http://fishbase.org/summary/653"
        },
        {
          "name": "Cetorhinus maximus",
          "url": "http://fishbase.org/summary/90"
        },
        {
          "name": "Deania calcea",
          "url": "http://fishbase.org/summary/670"
        },
        {
          "name": "Somniosus microcephalus",
          "url": "http://fishbase.org/summary/138"
        },
        {
          "name": "Centroscymnus coelolepis",
          "url": "http://fishbase.org/summary/662"
        },
        {
          "name": "Centroscymnus crepidater",
          "url": "http://fishbase.org/summary/663"
        },
        {
          "name": "Dalatias licha",
          "url": "http://fishbase.org/summary/669"
        },
        {
          "name": "Hexanchus griseus",
          "url": "http://fishbase.org/summary/637"
        },
        {
          "name": "Centroscyllium fabricii",
          "url": "http://fishbase.org/summary/656"
        }
      ]
    },
    {
      "name": "fishraylg",
      "type": "focal",
      "taxa": [
        {
          "name": "Amblyraja radiata",
          "url": "http://fishbase.org/summary/2565"
        },
        {
          "name": "Dipturus batis",
          "url": "http://fishbase.org/summary/2058"
        },
        {
          "name": "Raja clavata",
          "url": "http://fishbase.org/summary/2059"
        },
        {
          "name": "Rajella lintea",
          "url": "http://fishbase.org/summary/6422"
        }
      ]
    },
    {
      "name": "fishflatfishsmmd",
      "type": "focal",
      "taxa": [
        {
          "name": "Hippoglossoides platessoides",
          "url": "http://fishbase.org/summary/4239"
        },
        {
          "name": "Glyptocephalus cynoglossus",
          "url": "http://fishbase.org/summary/26"
        },
        {
          "name": "Lepidorhombus whiffiagonis",
          "url": "http://fishbase.org/summary/28"
        },
        {
          "name": "Phrynorhombus norvegicus",
          "url": "http://fishbase.org/summary/2360"
        }
      ]
    },
    {
      "name": "fishflatfishlg",
      "type": "focal",
      "taxa": [
        {
          "name": "Pleuronectes platessa",
          "url": "http://fishbase.org/summary/1342"
        },
        {
          "name": "Scophthalmus maximus",
          "url": "http://fishbase.org/summary/1348"
        },
        {
          "name": "Hippoglossus hippoglossus",
          "url": "http://fishbase.org/summary/1371"
        },
        {
          "name": "Reinhardtius hippoglossoides",
          "url": "http://fishbase.org/summary/516"
        }
      ]
    },
    {
      "name": "cephalopods",
      "type": "focal",
      "taxa": [
        {
          "name": "Todarodes sagittatus",
          "url": "http://fishbase.org/summary/57559"
        },
        {
          "name": "Rossia macrosoma",
          "url": "http://fishbase.org/summary/57462"
        },
        {
          "name": "Sepiola atlantica",
          "url": "http://fishbase.org/summary/57471"
        },
        {
          "name": "Neorossia caroli",
          "url": "http://fishbase.org/summary/57324"
        }
      ]
    },
    {
      "name": "zooplankton",
      "type": "background",
      "taxa": []
    },
    {
      "name": "phytoplankton",
      "type": "background",
      "taxa": []
    }
  ]
}
FIN-casey commented 7 years ago

@jhpoelen The inclusion of the extended list is not yet implemented. I would follow your suggestion in implementing this. Hopefully we won't encounter any limitation issues due to the large amount of data.

On Aug 16, 2017 5:53 AM, "Jorrit Poelen" notifications@github.com wrote:

On creating an osmose configuration for Iceland Shelf/Sea, I noticed that none of the extended taxa seem to make it into the request from the UI to the API (see below). I've attached the resulting osmose config here: osmose_config (6).zip https://github.com/jhpoelen/fb-osmose-bridge/files/1226555/osmose_config.6.zip

@FIN-casey https://github.com/fin-casey is there any thing the API needs to do to get access to the extended taxon list? If this is not yet implemented, I'd strongly suggest to use the method I suggested in #115 (comment) https://github.com/jhpoelen/fb-osmose-bridge/issues/115#issuecomment-311122588 .

Please let me know if there's anything I missed.

This is what got sent from UI to API:

{ "timeStepsPerYear": "12", "groups": [ { "name": "fishpelagicsm", "type": "focal", "taxa": [ { "name": "Mallotus villosus", "url": "http://fishbase.org/summary/252" }, { "name": "Benthosema glaciale", "url": "http://fishbase.org/summary/21" }, { "name": "Notoscopelus kroyeri", "url": "http://fishbase.org/summary/27753" } ] }, { "name": "fishpelagicmd", "type": "focal", "taxa": [ { "name": "Scomber scombrus", "url": "http://fishbase.org/summary/118" }, { "name": "Sardina pilchardus", "url": "http://fishbase.org/summary/1350" }, { "name": "Alosa fallax", "url": "http://fishbase.org/summary/5355" }, { "name": "Trachurus trachurus", "url": "http://fishbase.org/summary/1365" }, { "name": "Scomberesox saurus", "url": "http://fishbase.org/summary/1084" }, { "name": "Schedophilus medusophagus", "url": "http://fishbase.org/summary/1721" } ] }, { "name": "fishpelagiclg", "type": "focal", "taxa": [ { "name": "Salmo trutta", "url": "http://fishbase.org/summary/238" }, { "name": "Sebastes norvegicus", "url": "http://fishbase.org/summary/501" }, { "name": "Belone belone", "url": "http://fishbase.org/summary/47" }, { "name": "Brama brama", "url": "http://fishbase.org/summary/391" }, { "name": "Mola mola", "url": "http://fishbase.org/summary/1732" }, { "name": "Regalecus glesne", "url": "http://fishbase.org/summary/3267" } ] }, { "name": "fishdemersalsm", "type": "focal", "taxa": [ { "name": "Agonus cataphractus", "url": "http://fishbase.org/summary/36" }, { "name": "Ammodytes tobianus", "url": "http://fishbase.org/summary/38" }, { "name": "Liparis liparis", "url": "http://fishbase.org/summary/1378" }, { "name": "Callionymus maculatus", "url": "http://fishbase.org/summary/1807" }, { "name": "Ciliata mustela", "url": "http://fishbase.org/summary/1290" }, { "name": "Ciliata septentrionalis", "url": "http://fishbase.org/summary/1873" }, { "name": "Triglops murrayi", "url": "http://fishbase.org/summary/4146" }, { "name": "Liparis montagui", "url": "http://fishbase.org/summary/1379" }, { "name": "Micrenophrys lilljeborgii", "url": "http://fishbase.org/summary/9198" } ] }, { "name": "fishdemersalmd", "type": "focal", "taxa": [ { "name": "Eutrigla gurnardus", "url": "http://fishbase.org/summary/68" }, { "name": "Callionymus lyra", "url": "http://fishbase.org/summary/23" }, { "name": "Myoxocephalus scorpius", "url": "http://fishbase.org/summary/1329" }, { "name": "Pholis gunnellus", "url": "http://fishbase.org/summary/3804" }, { "name": "Lumpenus lampretaeformis", "url": "http://fishbase.org/summary/1380" }, { "name": "Boreogadus saida", "url": "http://fishbase.org/summary/319" }, { "name": "Enchelyopus cimbrius", "url": "http://fishbase.org/summary/1874" }, { "name": "Hyperoplus lanceolatus", "url": "http://fishbase.org/summary/1374" }, { "name": "Sebastes viviparus", "url": "http://fishbase.org/summary/1352" }, { "name": "Entelurus aequoreus", "url": "http://fishbase.org/summary/67" }, { "name": "Myxine glutinosa", "url": "http://fishbase.org/summary/2513" } ] }, { "name": "fishdemersallg", "type": "focal", "taxa": [ { "name": "Dicentrarchus labrax", "url": "http://fishbase.org/summary/63" }, { "name": "Melanogrammus aeglefinus", "url": "http://fishbase.org/summary/1381" }, { "name": "Pollachius virens", "url": "http://fishbase.org/summary/1343" }, { "name": "Chelon labrosus", "url": "http://fishbase.org/summary/2499" }, { "name": "Merluccius merluccius", "url": "http://fishbase.org/summary/30" }, { "name": "Molva molva", "url": "http://fishbase.org/summary/33" }, { "name": "Brosme brosme", "url": "http://fishbase.org/summary/51" }, { "name": "Polyprion americanus", "url": "http://fishbase.org/summary/349" }, { "name": "Petromyzon marinus", "url": "http://fishbase.org/summary/2530" }, { "name": "Molva dypterygia", "url": "http://fishbase.org/summary/1383" }, { "name": "Anarhichas lupus", "url": "http://fishbase.org/summary/2501" }, { "name": "Conger conger", "url": "http://fishbase.org/summary/301" }, { "name": "Anarhichas minor", "url": "http://fishbase.org/summary/3811" } ] }, { "name": "fishbathypelagicsm", "type": "focal", "taxa": [ { "name": "Myctophum punctatum", "url": "http://fishbase.org/summary/1328" }, { "name": "Maurolicus muelleri", "url": "http://fishbase.org/summary/754" }, { "name": "Argyropelecus hemigymnus", "url": "http://fishbase.org/summary/6968" }, { "name": "Argyropelecus olfersii", "url": "http://fishbase.org/summary/5097" } ] }, { "name": "fishbathypelagicmd", "type": "focal", "taxa": [ { "name": "Hoplostethus atlanticus", "url": "http://fishbase.org/summary/334" }, { "name": "Micromesistius poutassou", "url": "http://fishbase.org/summary/31" }, { "name": "Sebastes mentella", "url": "http://fishbase.org/summary/505" }, { "name": "Coryphaenoides mediterraneus", "url": "http://fishbase.org/summary/8175" }, { "name": "Coryphaenoides brevibarbis", "url": "http://fishbase.org/summary/27726" } ] }, { "name": "fishbathypelagiclg", "type": "focal", "taxa": [ { "name": "Aphanopus carbo", "url": "http://fishbase.org/summary/646" }, { "name": "Coryphaenoides rupestris", "url": "http://fishbase.org/summary/332" }, { "name": "Lampris guttatus", "url": "http://fishbase.org/summary/1072" }, { "name": "Trachipterus arcticus", "url": "http://fishbase.org/summary/3265" }, { "name": "Centrolophus niger", "url": "http://fishbase.org/summary/249" }, { "name": "Nemichthys scolopaceus", "url": "http://fishbase.org/summary/2660" }, { "name": "Coryphaenoides armatus", "url": "http://fishbase.org/summary/5127" } ] }, { "name": "fishbathydemersalmd", "type": "focal", "taxa": [ { "name": "Argentina silus", "url": "http://fishbase.org/summary/2700" }, { "name": "Argentina sphyraena", "url": "http://fishbase.org/summary/20" }, { "name": "Coryphaenoides guentheri", "url": "http://fishbase.org/summary/8176" }, { "name": "Echiodon drummondii", "url": "http://fishbase.org/summary/65" }, { "name": "Careproctus reinhardti", "url": "http://fishbase.org/summary/15500" }, { "name": "Bathysaurus ferox", "url": "http://fishbase.org/summary/11841" } ] }, { "name": "fishbathydemersallg", "type": "focal", "taxa": [ { "name": "Lophius piscatorius", "url": "http://fishbase.org/summary/716" }, { "name": "Alepocephalus bairdii", "url": "http://fishbase.org/summary/230" }, { "name": "Beryx decadactylus", "url": "http://fishbase.org/summary/1319" }, { "name": "Chimaera monstrosa", "url": "http://fishbase.org/summary/2503" }, { "name": "Synaphobranchus kaupii", "url": "http://fishbase.org/summary/9104" } ] }, { "name": "fishbenthopelagicsm", "type": "focal", "taxa": [ { "name": "Gasterosteus aculeatus", "url": "http://fishbase.org/summary/2420" }, { "name": "Ammodytes marinus", "url": "http://fishbase.org/summary/37" } ] }, { "name": "fishbenthopelagicmd", "type": "focal", "taxa": [ { "name": "Clupea harengus", "url": "http://fishbase.org/summary/24" }, { "name": "Merlangius merlangus", "url": "http://fishbase.org/summary/29" }, { "name": "Pagellus bogaraveo", "url": "http://fishbase.org/summary/890" }, { "name": "Trisopterus esmarkii", "url": "http://fishbase.org/summary/1023" }, { "name": "Cyclopterus lumpus", "url": "http://fishbase.org/summary/62" }, { "name": "Coelorinchus caelorhincus", "url": "http://fishbase.org/summary/1726" } ] }, { "name": "fishbenthopelagiclg", "type": "focal", "taxa": [ { "name": "Gadus morhua", "url": "http://fishbase.org/summary/69" }, { "name": "Salmo salar", "url": "http://fishbase.org/summary/236" }, { "name": "Salvelinus alpinus", "url": "http://sealifebase.org/summary/alpinus" }, { "name": "Macrourus berglax", "url": "http://fishbase.org/summary/331" }, { "name": "Pollachius pollachius", "url": "http://fishbase.org/summary/34" }, { "name": "Phycis blennoides", "url": "http://fishbase.org/summary/1340" }, { "name": "Anarhichas denticulatus", "url": "http://fishbase.org/summary/3810" }, { "name": "Argyrosomus regius", "url": "http://fishbase.org/summary/418" }, { "name": "Notacanthus chemnitzii", "url": "http://fishbase.org/summary/2661" } ] }, { "name": "fishreefassociatedsm", "type": "focal", "taxa": [ { "name": "Lebetus scorpioides", "url": "http://fishbase.org/summary/456" } ] }, { "name": "fishsharksmmd", "type": "focal", "taxa": [ { "name": "Etmopterus princeps", "url": "http://fishbase.org/summary/683" }, { "name": "Etmopterus spinax", "url": "http://fishbase.org/summary/687" } ] }, { "name": "fishsharklg", "type": "focal", "taxa": [ { "name": "Lamna nasus", "url": "http://fishbase.org/summary/88" }, { "name": "Squalus acanthias", "url": "http://fishbase.org/summary/139" }, { "name": "Galeorhinus galeus", "url": "http://fishbase.org/summary/4642" }, { "name": "Centrophorus squamosus", "url": "http://fishbase.org/summary/653" }, { "name": "Cetorhinus maximus", "url": "http://fishbase.org/summary/90" }, { "name": "Deania calcea", "url": "http://fishbase.org/summary/670" }, { "name": "Somniosus microcephalus", "url": "http://fishbase.org/summary/138" }, { "name": "Centroscymnus coelolepis", "url": "http://fishbase.org/summary/662" }, { "name": "Centroscymnus crepidater", "url": "http://fishbase.org/summary/663" }, { "name": "Dalatias licha", "url": "http://fishbase.org/summary/669" }, { "name": "Hexanchus griseus", "url": "http://fishbase.org/summary/637" }, { "name": "Centroscyllium fabricii", "url": "http://fishbase.org/summary/656" } ] }, { "name": "fishraylg", "type": "focal", "taxa": [ { "name": "Amblyraja radiata", "url": "http://fishbase.org/summary/2565" }, { "name": "Dipturus batis", "url": "http://fishbase.org/summary/2058" }, { "name": "Raja clavata", "url": "http://fishbase.org/summary/2059" }, { "name": "Rajella lintea", "url": "http://fishbase.org/summary/6422" } ] }, { "name": "fishflatfishsmmd", "type": "focal", "taxa": [ { "name": "Hippoglossoides platessoides", "url": "http://fishbase.org/summary/4239" }, { "name": "Glyptocephalus cynoglossus", "url": "http://fishbase.org/summary/26" }, { "name": "Lepidorhombus whiffiagonis", "url": "http://fishbase.org/summary/28" }, { "name": "Phrynorhombus norvegicus", "url": "http://fishbase.org/summary/2360" } ] }, { "name": "fishflatfishlg", "type": "focal", "taxa": [ { "name": "Pleuronectes platessa", "url": "http://fishbase.org/summary/1342" }, { "name": "Scophthalmus maximus", "url": "http://fishbase.org/summary/1348" }, { "name": "Hippoglossus hippoglossus", "url": "http://fishbase.org/summary/1371" }, { "name": "Reinhardtius hippoglossoides", "url": "http://fishbase.org/summary/516" } ] }, { "name": "cephalopods", "type": "focal", "taxa": [ { "name": "Todarodes sagittatus", "url": "http://fishbase.org/summary/57559" }, { "name": "Rossia macrosoma", "url": "http://fishbase.org/summary/57462" }, { "name": "Sepiola atlantica", "url": "http://fishbase.org/summary/57471" }, { "name": "Neorossia caroli", "url": "http://fishbase.org/summary/57324" } ] }, { "name": "zooplankton", "type": "background", "taxa": [] }, { "name": "phytoplankton", "type": "background", "taxa": [] } ] }

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/jhpoelen/fb-osmose-bridge/issues/115#issuecomment-322600357, or mute the thread https://github.com/notifications/unsubscribe-auth/AReVf3fQBSpslBUkaUwbJ1knzFOxnoLXks5sYhNEgaJpZM4Nm9A2 .

jhpoelen commented 7 years ago

@FIN-casey Thanks for your comment that helps explain why @agruss2 does not see the expected behavior. Assigning issue to you.

FIN-casey commented 7 years ago

@agruss2 @jhpoelen , I implemented the inclusion of "implicit" species in the data sent to the API. The implementation is now in the wizard. Can you please verify and test the changes?

Thanks. Please take note that there is a delay in showing the message "Preparing configuration..." after clicking Submit.

agruss2 commented 7 years ago

@FIN-casey @jhpoelen I just ran a test and I can confirm that the implementation is successful; many thanks Casey! Therefore, I am going to close the present issue. However, please note that I see some potential areas of improvement. Therefore, I will soon create new GitHub issues to address some minor problems.

jhpoelen commented 7 years ago

@FIN-casey @agruss2 good the hear that the parameter estimation using the implicit groups is now working as expected. Thanks for fixing and testing!

agruss2 commented 7 years ago

@jhpoelen @FIN-casey I am reopening this issue, because I am still having a few NAs here and there for focal functional groups. Therefore, I think that, for each functional group, we should increase the number of species that the bridge between FishBase/SeaLifeBase and OSMOSE considers in addition to the species making up the functional groups. How many additional species does the bridge currently consider for each focal functional group? How much could we increase that number without making the bridge crash? Many thanks.

jhpoelen commented 7 years ago

@agruss2 the API does not control the amount of taxa that are associated with focal/background functional groups. This is why I consider this a UI issue.

Same applies to #114 .