osmose-model / osmose-web-api

Web service that generates Osmose configuration files from data sources like Fishbase and SeaLifeBase. Used by https://www.config.osmose-model.org .
MIT License
2 stars 2 forks source link

Minimum and maximum predator/prey size ratios #32

Closed agruss2 closed 6 years ago

agruss2 commented 8 years ago

@jhpoelen @FIN-JBarile @Dengaloo It is necessary to define minimum and maximum predator/prey size ratios for the juveniles (immature individuals) and adults (mature individuals) of the different focal functional groups represented in an OSMOSE model (NB: the predator/prey size ratios of juveniles and adults of a given focal functional group can be identical if this makes sense from an ecological standpoint). In the "OSMOSE parameters and Comments" word document available in Dropbox, Skit and Deng wrote the following about minimum and maximum predator/prey size ratios : "Needs more discussion; what are the parameters? The MORPHDATA (morphology) and/or MORPHMET (morphometrics) tables may be useful". I had a look at the MORPHDATA and MORPHMET, but it is currently unclear to me how we could estimate minimum and maximum predator/prey size ratios through the bridge between FishBase/SeaLifeBase and OSMOSE. I would like all of us to discuss this issue further during our next phone call.

jhpoelen commented 8 years ago

@agruss2 nothing I can add at this point. Looking forward to the discussion.

jhpoelen commented 8 years ago

@FIN-JBarile is planning to work with @Dengaloo about this tomorrow (Aug 9, 2016 US)

FIN-JBarile commented 8 years ago

@agruss2 @Dengaloo

Per Skype call with Deng on 9/10 August, it is suggested to use the available parameters in the Life history tool of FishBase, specifically Length at maturity (Lm), Length infinity (Linf) and possibly the growth coefficient (K).

Length at maturity (Lm) would be the minimum length of an adult, and anything less would apply to juveniles.

Length infinity (Linf) would be the upper limit for adult length

See for example, http://www.fishbase.org/PopDyn/KeyFactsSummary_1.php?ID=120&GenusName=Scomberomorus&SpeciesName=cavalla&vStockCode=134&fc=416

For more info about the parameters of life history data, please see http://www.fishbase.org/manual/Key%20Facts.htm. Default values provided by this tool are medians of different studies for a given species. In the absence of data, an equation is applied as described in above document. Practically, there are parameter values for all species in FishBase.

Default values presented per species are stored in the table MATRIX in FishBase.

Alternatively, the size parameters can be obtained as median of available studies specific to an ecosystem/locality, e.g. http://www.fishbase.org/PopDyn/PopGrowthList.php?ID=120&GenusName=Scomberomorus&SpeciesName=cavalla&fc=416 In this case, we use the POPGROWTH table.

agruss2 commented 8 years ago

@Dengaloo @FIN-JBarile Thanks a lot for having worked on this issue ! This is very interesting information, but I am not sure how to use this to get estimates of minimum and maximum predator/prey size ratios for OSMOSE. Minimum and maximum predator/prey size ratios are used in OSMOSE to determine size adequacy between a predator and a potential prey item; minimum and maximum predator/prey size ratios are somehow the equivalent of the "gape size" parameters used in Atlantis. They are a way to control what juveniles and adults of focal functional groups will eat. For instance, if adult gag grouper has a maximum predator/prey size ratio of 4 and a minimum predator/prey size ratio of 16, then adult gag grouper will eat any prey item that is at least 4 times smaller than it and at most 16 times smaller than it. A few months ago, Deng mentioned that predator/prey size ratios could be estimated by FishBase/SeaLifeBase through a calculation of gape size; gape size can be obtained as a percentage of total length (see morphometrics in FishBase/SeaLifeBase). ===> I suggest that Deng, Skit and I discuss this issue further during our next phone call (i.e., on August, 15th 2016) or short after.

Dengaloo commented 8 years ago

Yes, I agree with Arnaud, better that we discuss live so I can show you how this can be done using the length at first maturity (as an indicator of juvenile and adult lengths) and the % measurements of body parts.

agruss2 commented 8 years ago

@Dengaloo Hi Deng,

Sorry for having taken so long to come back to you.

I compiled a spreadsheet listing the minimum and maximum predator/prey size ratios fed into the different OSMOSE models that have been published so far. I uploaded this spreadsheet to our shared Dropbox folder (have a look in "RESTORE Act project/OSMOSE material"). Could you please derive a predictive model for minimum predator/prey size ratio and another predictive model for maximum predator/prey size ratio, using pertinent variables (e.g., trophic level, maximum length, and food type) as independent variables? This exercise will be similar to that you conducted in Palomares and Pauly (1998) for deriving a predictive model for Q/B.

A few comments: (1) Let’s fit two models (i.e., 1 for min. pred./prey size ratio; and 1 for max. pred./prey size ratio) instead of four models (i.e., 1 for min. pred./prey size ratio for juveniles; 1 for max. pred./prey size ratio for juveniles; 1 for min. pred./prey size ratio for adults; and 1 for max. pred./prey size ratio for adults). In other words, the juveniles and adults of the different functional groups listed in the spreadsheet that I uploaded to Dropbox will be considered as individual data points in the models. (2) I think that the OSMOSE model for the Strait of Georgia also uses min. and max. pred./prey size ratios. I am going to contact Caihong Fu to check this. If the OSMOSE model for the Strait of Georgia indeed used min. and max. pred./prey size ratios, then we would have more data points to fit models. (3) In case the models we fit do not yield satisfactory results, then we will have to develop another methodology to get estimates of min. and max. pred./prey size ratios for a new OSMOSE model. A possibility would be to use some kind of fuzzy logic, which would use the information provided in the spreadsheet attached to this email to derive estimates of min. and max. pred./prey size ratios. (4) This work will be useful not only to OSMOSE modelers, but also to any scientist working with models/tools that require insights into the size range of the potential prey of different predators (e.g., size spectrum models). Therefore we might consider writing a short paper to report this work (e.g., a short communication).

We can meet on Skype any time to discuss all this further, if you want. Many thanks.

agruss2 commented 8 years ago

@Dengaloo I do not know if you have made any progress on the models to predict minimum and maximum predator/prey size ratios. I am starting to be really concerned about the slow rate at which we are making progress in the entire project. We need to provide results to the grant agency on time. Therefore, if you think that you will not have time to work on the models predicting minimum and maximum predator/prey size ratios, then please let me know and I will handle this myself. However, if I need to work on the models predicting minimum and maximum predator/prey size ratios myself, then we should meet on Skype soon, so that you clearly explain me where I could get all the necessary information for fitting the models. Note that, to facilitate the development of models predicting minimum and maximum predator/prey size ratios, I reworked the “Pred_prey_size_ratios_OSMOSE_models” spreadsheet (attached here). Pred_prey_size_ratios_OSMOSE_models.xlsx

FIN-JBarile commented 8 years ago

Under Deng’s supervision, a multiple regression analysis is currently being carried out, based on the data provided by Arnaud.

Independent variables used are trophic level, max length and food type (main food) as available in FishBase/SeaLifeBase; while given pred/prey ratio is dependent variable. Results will be a predictive model that will be used to estimate pred/prey ratios as discussed by Deng and Arnaud.

@agruss2, can you confirm if this file should be the basis? Pred_prey_size_ratios_OSMOSE_models.xlsx

This file was attached to your e-mail on 5 September. Is this dataset based on the file you sent on 24 August (data by Ghassen Halouani)?

Meantime, can the existing pred/prey ratios in your file above be used as dummy data for programming purposes? That way the programming can move forward while we await completion of the analyses.

@Dengaloo

agruss2 commented 8 years ago

@FIN-JBarile @Dengaloo Sounds very good; many thanks. I confirm that the following file should be the basis of the analyses undertaken under Deng's supervision: Pred_prey_size_ratios_OSMOSE_models.xlsx In the meantime, the existing pred/prey ratios in the file above can be used as dummy data for programming purposes, so that we can make progress in every way we can. However, the purpose of deriving a predictive model for min. and max. pred./prey size ratios is to integrate some code in @jhpoelen's API. If this API is not enhanced to communicate all the information that the wizard needs to provide OSMOSE parameters to the user, then it will not be possible to complete our project.

jhpoelen commented 8 years ago

@agruss2 @FIN-casey @Dengaloo

The mapping between fishbase api and osmose parameters can be found here: https://github.com/jhpoelen/fb-osmose-bridge/blob/master/src/main/resources/com/github/jhpoelen/fbob/fishbase-mapping.csv

If the max/min size ratios are accessible through fishbase api , connecting this valuable information can be as simple as filling in the "?" in the csv file.

However, if some algorithmic conversation needs to happen to convert or combine values extracted from the fishbase api, some trivial java code needs be implemented.

Hope this helps.

FIN-JBarile commented 8 years ago

@jhpoelen @agruss2 @Dengaloo

The intention is to store the min/max pred/prey size ratios, derived from the model, in FishBase through the ESTIMATE table. The fields have yet to be added to the table. Will add details for the mapping of parameters when ready. However, the ESTIMATE table is not in the fishbase.api yet. Will need to request rOpenSci to include.

jhpoelen commented 8 years ago

@FIN-JBarile sounds good to me. I am sure that the friendly folks at rOpenSci like Scott @sckott and Carl @cboettig would be happy to help with this.

Might be a bit of a stretch but . . . why not have fishbase api automatically upgrade to a new fishbase/sealifebase version whenever one is available? All you'd have to do is publish the archive somewhere where fishbase api can find it.

sckott commented 8 years ago

why not have fishbase api automatically upgrade to a new fishbase/sealifebase version whenever one is available?

From my understanding, new versions of the database are very infrequent (like many months apart), so we just do the upgrade manually.

Also, the SQL dump isn't public, making it impossible to automate

jhpoelen commented 8 years ago

@sckott thanks for chiming in!

FIN-JBarile commented 8 years ago

@jhpoelen thanks for connecting with rOpenSci folks. Will e-mail request to Carl and Scott.

Dengaloo commented 7 years ago

We are still looking at how to provide intelligent estimates of this parameter. I am testing a hypothesis presented in Ursin (1973) using the weights instead of lengths of predators and prey. We are seeing a very good relationship using weights, but will send in a description of this estimation once I have confirmed that the results we are getting are viable.

agruss2 commented 7 years ago

@Dengaloo Thanks a lot, this sounds excellent. Looking forward to reading the description of this work.

Dengaloo commented 7 years ago

@agruss2 : actually I sent a preliminary analysis to Daniel and asked his opinion if this is a viable empirical model, following Ursin (1973). Daniel said that it was beautiful and that we should write a paper about it! Now, that was only for juv and adult fish predators preying on juv and adult fish prey. So only fish on fish, and the relationship between log weight of prey vs log weight of predator is very good, n=631 pairs in FishBase, r2=0.48. Ursin (1973) suggested that this can be used as a measure of predator/prey size ratio. So, I want to try with non fish data as well. We are still extracting the data. In SeaLifeBase, we have lengths, so we are turning these into weights, from L/W relationships. Then the data manipulation also involves grouping predators into adults preying on adult prey, juvenile predator preying on juvenile prey, and all those combinations. Then we look at the relationships. So, give me some more time please. Thanks.

agruss2 commented 7 years ago

@Dengaloo This sounds really terrific; many thanks!

Dengaloo commented 7 years ago

@agruss2 : I think I should have a working model by end of this week. What takes time is the grouping of the predator weights into 0.1 log units. But, we have a working template now, and I have instructed the SLB research assistants on it. I am hitting two birds with one stone, also teaching the SLB RAs to write queries. So next time, when I am in Vancouver, I will be able to ask for a data extraction and they will be able to write the necessary queries for it.

agruss2 commented 7 years ago

@Dengaloo This is really great! We should think whether this work on the predator/prey size ratios should be published in the paper presenting the bridges (the deliverable of the RESTORE Act project) or whether it should be published in another paper; I leave the decision to you.

Dengaloo commented 7 years ago

@agruss2 : I think we should present this as one of the empirical models that we will come up in this project, that is of data based on FB and SLB. I would rather have something that accumulates all of these empirical models in a paper and use this also as a reference for tools that we would like to implement in SLB and FB. I am also now working on the Bayesian prior for length-weight relationships for invertebrates - we now have more than 3,000 a and b data pairs from independent studies. Massive data sets, but which now need to be checked for viability. I am having the SLB research assistants help me with each one of these data sets so I can provide these models to the bridge. So exciting times right now here at SLB.

agruss2 commented 7 years ago

@Dengaloo Please let me know if I understood correctly: you think that we should report this work on the predator/prey size ratios in the paper for the RESTORE Act project, but that we should also publish this in a future paper that will provide different empirical models that have recently been developed at SLB. Is that correct?

agruss2 commented 7 years ago

@Dengaloo I extended the deadline of this issue to February, 19th. Many thanks!

Dengaloo commented 7 years ago

@agruss2 : thanks, we are still working on making the underlying data clear of errors - encoding errors which I discovered while working with the data. almost there. will give this over by end of week.

agruss2 commented 7 years ago

@Dengaloo Perfect, thanks!

agruss2 commented 7 years ago

@Dengaloo I am extending the deadline of this issue to March, 5th. Many thanks!

agruss2 commented 7 years ago

@Dengaloo There have not been any discussions on the estimation of parameters for OSMOSE between you and me during the past weeks. Hopefully, these discussions will resume next week, so that I can open new issues for Jorrit to estimate values for the remaining OSMOSE parameters. I am extending the deadline of this issue again. Thanks a lot.

Dengaloo commented 7 years ago

@agruss2 : Discussing with @FIN-JBarile this afternoon on how to provide this parameter for the bridge. Easiest way is via an empirical equation which needs to be proven first. This equation has X as the log weight of the predator and Y as the log weighted average weight of prey. Because we have data for juveniles, adults and juveniles and adults combined, it would be prudent to have separate analyses for these three cases, including a fourth case with all of the data combined. So for a representative set of species (I think we have >1000 species represented in this data set, but I will confirm with Jenks), we will organize the data into classes of predator sizes and prey sizes. For each predator class size, there would be a number of prey being consumed, which in turn can be arranged as prey class sizes for that predator class size. We would need to get the weighted average size of the prey for that predator class size. Thus, we will have a final data set that will have the mid class size of the predator and the weighted average size of the prey for n prey items. If we find that the predator size and the prey size are significantly correlated, then we will be able to predict the prey size for a given predator size. We can also get the min and max prey sizes by predator class size and get predator/prey ratios from those. Note that I get this idea from Ursin (1972) who writes and I quote: "prey size preference when prey animals are distributed in the way most of them are [...]" an assumption that "on encounter with a prospective prey the predator evaluates the size of the prey relative to its own size and that the evaluation is symmetrical on a percentage scale, so that, e.g., 'half as big' and 'twice as big' differ by the same amount from 'just fine'. " end of quote. So, by making the histogram analysis, we are assuming normality in the choice of prey size.

Is this acceptable?

Note that a preliminary analysis came up with a significant correlation between predator and average prey size. I wanted to express prey size instead as a weighted average because in some instances, there is quite a spread but with a clear trend towards a certain size range.

Also note that we are using maximum sizes for the species to make this a species based analysis and not a population based analysis. Basically assuming that predator species, in a general sense, will pick on prey that are on average of a certain size.

Let me know if there is a loop hole in this logic. And send us your comments please.

agruss2 commented 7 years ago

@Dengaloo @FIN-JBarile
I agree that an empirical equation relating the log weight of the predator to the log weighted average weight of prey is the way to go. I also agree that it is good to derive equations for juveniles, adults, and juveniles and adults combined, since, in some cases, it will not make sense or it will not be possible to derive equations for different life stages. It is also acceptable to assume normality in the choice of prey size. Moreover, it makes sense to make the analysis on a species basis rather than a population basis, due to the functioning of the bridge between FishBase/SeaLifeBase and OSMOSE, which considers “reference species” in turn to extract parameter values so as to define a value for a given simple or composite parameter for a given functional group.
So, things are going in the right direction here. Please keep me posted about your progress.

Dengaloo commented 7 years ago

@agruss2 : ok so we have established the relationship between maximum predator size and maximum prey size by grouping the predator maximum weights (g;log) in 0.1, 0.5 and 1 log unit class intervals, and for each predator weight class interval we nested histograms of prey maximum weight (also in 0.1, 0.5 and 1 log unit intervals). The result gives us a series of histograms that can be used to get the weighted average prey maximum weight log units per log unit of predator weight class size. here is an example, using 1 log unit class intervals:

image

given average predator weights, we can estimate the average prey weights, and get the ratio.

will this work? I can send you the Excel file in an email

note that the X-axis is nested in two: the bottom numbers spaced out in longer lines are the predator weight intervals (g;log); and the inner numbers are the prey weight intervals (g;log). the vertical axis is frequency per class interval. this graph shows that each predator weight class size corresponds to a prey weight peak, assuming that the distribution of prey weights per predator weight is normally distributed.

agruss2 commented 7 years ago

@Dengaloo This looks great; many thanks! However, while I think that I understand your analyses on predator weights and prey weights, I am unclear how you will use the results of these analyses to derive minimum and maximum predator/prey size ratios for the juveniles and adults of the fish species represented in FishBase. Could you please explain this to me in detail? Thanks a lot.

P.S.: If I understand well, we will not be able to derive minimum and maximum predator/prey size ratios for invertebrate species from SeaLifeBase data?

P.S. 2: Yes, please email me the Excel file.

Dengaloo commented 7 years ago

@agruss2 : if our rule for juvenile fish is 50% of Lm, then our minimum weight will be the corresponding of that length; our maximum weight would be the Lmax.

We can estimate minimum and maximum pred/prey size ratio for invertebrates as well, using the above mentioned rule. In cases where there are no growth parameters, then we might want the user to define these parameters.

The relationship I posted in the last comment is for all fishes and invertebrates. That is why I opted to use weight, because we can compare across species groups if we used weights instead of lengths.

Dengaloo commented 7 years ago

@agruss2 : I want to write a paper on this plus on the auximetric grid for all marine animals, and a Bayesian rule base for each iinvertebrate group. I will work on it this June and submit it by end of the month to Marine Biology or some other journal that still accepts such biological comparisons. Is this acceptable?

Dengaloo commented 7 years ago

@agruss2 : bayesian rule base on length-weight relationships that is.

Dengaloo commented 7 years ago

@agruss2 @FIN-JBarile : here is the relationship with raw data, using predator weight from maximum length of species, and simply averaging the prey weight from Lmax of consumed species

image

Dengaloo commented 7 years ago

@agruss2 @FIN-JBarile : here is the relationship for adult and juv/adult predators

image

Dengaloo commented 7 years ago

@agruss2 @FIN-JBarile : and finally, the relationship for larvae and recruits/juv

image

Dengaloo commented 7 years ago

@agruss2 @FIN-JBarile : in an attempt to decrease the uncertainty caused by the spread of the first three relationships, I obtained the weighted average prey weight from the histograms in a previous comment and obtained the linear relationship (note that the curve looks like a polynomial, but I don't want to go that way. I was hoping that it would look like a VBGF curve, that would imply that after a certain maximum size, the size of prey would not matter anymore - which is what I had read from the histogram. But that does not seem like the case as the progression seems to increase a little bit more:

image

agruss2 commented 7 years ago

@Dengaloo Sorry for the late reply, I was travelling to France. To answer your first comment: (1) Could you please apply your methodology to the juveniles and adults of two or three given species of the Gulf of Mexico, so that I can gauge its performance? (2) I think that it would be better to integrate minimum and maximum predator/prey size ratio estimates in a given FishBase Table (CC: @FIN-JBarile). This would increase the chances of the bridge between FishBase/SeaLifeBase and OSMOSE to provide minimum and maximum predator/prey size ratios for the different focal functional groups defined for the OSMOSE model under consideration. What do you think?

agruss2 commented 7 years ago

@Dengaloo To answer your second comment: I think that it is an excellent idea to have a paper reporting the estimation of min. and max. predator/prey size ratios, as well as the auximetric grid for all marine animals and the Bayesian rule base for each invertebrate group. You should also feel free to provide in this paper the other relationships that you established within the RESTORE Act project so that we could derive estimates for some OSMOSE parameters. The paper you are envisioning will greatly facilitate the writing of the paper on the bridge between FishBase/SeaLifeBase and OSMOSE (current target journal: Environmental Modelling & Software), since we will then not have to describe in great detail how we estimated critical predation efficiencies, min. and max. predator size ratios, etc., but instead will refer to your paper. The timing is also excellent, since we should start working on the paper on the bridge between FishBase/SeaLifeBase and OSMOSE on July, 1st the latest. If this is acceptable to you, I'd be very happy to be a co-author of your paper.

agruss2 commented 7 years ago

@Dengaloo (1) Could you please apply your methodology to the juveniles and adults of four or five given species of the Gulf of Mexico and send me your results, so that I can gauge the performance of your methodology? (2) I think that it would be better to integrate minimum and maximum predator/prey size ratio estimates in a given FishBase Table (CC: @FIN-JBarile). This would increase the chances of the bridge between FishBase/SeaLifeBase and OSMOSE to provide minimum and maximum predator/prey size ratios for the different focal functional groups defined for the OSMOSE model under consideration. What do you think?

FIN-JBarile commented 7 years ago

@agruss2 Submitting the ESTIMATE table which contains estimates of min and max predator/prey size ratios of 3548 fish species. Maybe Jorrit can directly update his copy of the ESTIMATE table of FishBase using this attachment while we haven't provided an update to rOpenSci for the FishBase API yet.

Estimates for non-fish will follow.

agruss2 commented 7 years ago

@FIN-JBarile Thanks a lot for the update. Please let me know once the estimates for non-fish are ready. I will then assess the pred./prey size ratios produced for fish and non-fish, and decide whether the present issue issue can be closed and a dedicated API issue can be opened for Jorrit. Talk soon and many thanks for everything once again.

FIN-JBarile commented 7 years ago

@agruss2 Here is the ESTIMATE table for non-fish, with estimates of min and max predator/prey size ratios of 223 species.

Here are lists that contain details of the species with ratios for FB and SLB.

cc: @Dengaloo

agruss2 commented 7 years ago

@FIN-JBarile @Dengaloo Sorry for the late reply. Could you please generate new ESTIMATE and ESTIMATE_SLB tables with species Latin names, so as to facilitate my analysis of the files? I have a major comment: I think that fitting one single linear relationship for fish and one single linear relationship for non-fish will not give us relevant predator/prey size ratios. If we do that, we will obtain predator/prey size ratios smaller than 1 for the smallest fish and non-fish species, which is not relevant. To address this issue, we could use stepwise regression, so as to have different relationships for the different size classes of fish and the different size classes of non-fish. Then, we would be able to develop, for example, linear relationships with a steeper slope for the smallest fish and non-fish species, which would yield larger and more relevant predator/prey size ratio estimates. However, I will need to dig more into the ESTIMATE and ESTIMATE_SLB tables before being able to provide more comprehensive thoughts. Therefore, again, could you please generate new ESTIMATE and ESTIMATE_SLB tables with species Latin names? Many thanks and talk soon.

FIN-JBarile commented 7 years ago

@agruss2 I provided above the lists that have the Latin names for species with ratios FB and SLB above.

Anyway, here are the ESTIMATE tables for FB and SLB with Latin names. ESTIMATE_v2.txt ESTIMATE_SLB_v2.txt

cc: @Dengaloo

agruss2 commented 7 years ago

@Dengaloo (CC: @FIN-JBarile) Deng - Could you please email me some text (and figures) explaining the calculation of minimum and maximum predator/prey size ratios in detail? This would help me to offer suggestions so that we do not get predator/prey size ratios smaller than 1. Many thanks !!

jhpoelen commented 7 years ago

To help move things along, I've created a patch release for the fishbase/sealifebase cache with the estimate tables for fishbase and sealifebase. See https://github.com/jhpoelen/fishbase_archiver/releases/tag/v0.2.1-patch for more information.

FIN-JBarile commented 6 years ago

@agruss2 Here are the ESTIMATE tables for fish and non-fish, with estimates of min and max predator/prey size ratios (3545 for fish and 222 for non-fish). Latin names are included as well.

Please find the equation used to get the size ratios in the file below. This document was prepared Jenks with Deng. Note that the equations are different from the previous ones used.

OSMOSE_Documentation_Methods_PredPrey_sizeratio.docx

Note: ESTIMATE tables also contain AgeMin and AgeMax parameters for Issue #23

cc: @Dengaloo