Making sure that the OSMOSE API extracts data from SeaLifeBase tables for non-fish species

[agruss2]

@jhpoelen @FIN-JBarile and @QQ-Sortiz ran some tests (reported below) and, based on the results of these tests, we concluded that, currently, the OSMOSE API may not extract data from SeaLifeBase tables to produce OSMOSE parameter estimates for non-fish (i.e., invertebrate) species. This may explain why we get only NA's and default values for all the invertebrate groups represented in the OSMOSE-WFS model. Therefore, could you please check the OSMOSE API and make the necessary changes in the OSMOSE API, if need be? When you are done, please let me know so that I can run a new test with the web application. Many thanks!

---

Here is what Skit told me on August, 28th 2018, based on which we are suspecting the aforementioned issue with the OSMOSE API: We tried the following cases and verified with the code on both the Wizard UI and OSMOSE API.

Case 1:

Run for Gulf of Mexico.

Remove all fish functional groups.

Retain all non-fish functional groups.

See functionalgroups.csv particularly species url. -> uses URL of FB for non-fish species.

Case 2:

Run for Gulf of Mexico.

Include only the fish functional group with Lutajanus campechanus as species.

Retain all non-fish functional groups.

See functionalgroups.csv particularly species url. -> uses URL of FB for non-fish species. AND values for non-fish are either NA’s or defaults in osm_param-species.csv.

Case 3:

Same as Case 2 but ADD non-fish group/species Callinectes sapidus as focal group.

See functionalgroups.csv particularly species url. -> uses URL of FB for fish species while it uses URL of SLB for C. sapidus. AND values for C. sapidus are either NA’s or defaults in osm_param-species.csvs.

However, C.sapidus has data for maturity, length-weight and growth (k, Linf) and longevitywild. Longevitywild data can be verified in species_sealifebase.tsv.gz in fishbase_archiver v0.2.1 , in which Miel has uploaded the tables with updated content recently, e.g. longevitywild of C. sapidus is a recent entry.

Case 4:

Run for Antartic.

Accept groups and species in default list.

See functionalgroups.csv particularly species url. -> uses URL of FB for non-fish species.

Case 5

OSMOSE config file generated by AGruss for West Florida Shelf, as described in https://github.com/jhpoelen/fb-osmose-bridge/issues/173#issuecomment-413613992.

aFunctional groups were recreated, seemingly similar to the case of adding a group. Behaves as in Case 3.

With above cases, we found that there is a second issue on top of the NA's, i.e. for cases 1,2 and 4, the Wizard UI outputs the functional_groups json with incorrect URL's for non-fish species. It uses that of FishBase. Miel has corrected that part already. But that's still not the cause for the NA's, which are evident only in non-fish. Our NA's here are somewhat false NA's as illustrated in the case of Callinectes sapidus, i.e. species has data but is rendered with NA's. See Case 3.

We know from ValueFactoryFishbaseCache.java that the version of the fishbase_archiver used by the API is v0.2.1. So that's where Miel applied the updated data for some tables that you pointed out with NA's issue. It is also in the same code where the data source is handled. As far as we understand, there is provision only for FishBase and none for SeaLifeBase. Although the SLB tables are in the fishbase_archiver, they don't seem to be fetched in the ValueFactory code. Again, that's our understanding of the flow of that part of the OSMOSE API.

[jhpoelen]

@agruss2 @FIN-JBarile @QQ-Sortiz Thanks to the detailed information you provided, I was able to reproduce and fix the issue. Please confirm that the behavior is as expected.

[agruss2]

@jhpoelen Thanks a lot for having worked on this. I just ran a test where I queried parameter estimates for the OSMOSE model for the West Florida Shelf ("OSMOSE-WFS"). Here is the "osmose_config.zip" file that the web application provided me with: osmose_config.zip As you can see, the web application provided me with default values and NA's for all parameters for all functional groups. Do you know what is causing this? Could you please fix the problem and then let me know when you think it is fixed so that I can run a new test with the web application? Many thanks!

[jhpoelen]

I prepared another fix to attempt to address the described issue. As I was testing the outcomes, I ran across #176 and #177 . The former I addressed myself, and #177 appears minor, but might need some attention from @FIN-JBarile @QQ-Sortiz .

@agruss2 please confirm that the desired behavior is now available.

[agruss2]

@jhpoelen I ran a test today, where I queried parameter estimates for the Gulf of Mexico for the OSMOSE model of the West Florida Shelf (“OSMOSE-WFS”). The majority of the issues we were previously facing were solved, which is really great. However, there are a few issues that remain to be solved. Therefore, I am going to close the present issues and open two new GitHub issues. Many thanks for all your great work and talk soon!