iobis / obis-issues

Repository for all OBIS related issues and feature requests
4 stars 3 forks source link

Marine species reported as "non-marine" is actually not-accepted #83

Open auspex opened 5 years ago

auspex commented 5 years ago

On https://portal.obis.org/table/nonmarine?datasetid=0b6c1359-22b7-46b6-bfb0-b67afaf7f245, "Ceratium candelabrum" is reported as non-marine, but WoRMS says it's Marine, but is not accepted

pieterprovoost commented 5 years ago

@auspex @wardappeltans @bart-v The problem is not with the unaccepted status but with the environment flags of the accepted taxon. These must have been fixed by WoRMS very recently, because this is this is what was fetched from the WoRMS API (not marine, not brackish):

{
  "url": "http://www.marinespecies.org/aphia.php?p=taxdetails&id=841193",
  "lsid": "urn:lsid:marinespecies.org:taxname:841193",
  "rank": "Species",
  "class": "Dinophyceae",
  "genus": "Tripos",
  "order": "Gonyaulacales",
  "family": "Ceratiaceae",
  "phylum": "Myzozoa",
  "status": "accepted",
  "AphiaID": 841193,
  "kingdom": "Chromista",
  "citation": "Guiry, M.D. & Guiry, G.M. (2018). AlgaeBase. World-wide electronic publication, National University of Ireland, Galway (taxonomic information republished from AlgaeBase with permission of M.D. Guiry). Tripos candelabrus (Ehrenberg) F.Gómez, 2013. Accessed through: World Register of Marine Species at: http://www.marinespecies.org/aphia.php?p=taxdetails&id=841193 on 2018-07-20",
  "isMarine": 0,
  "modified": "2015-06-26T14:00:51Z",
  "authority": "(Ehrenberg) F.Gómez, 2013",
  "isExtinct": null,
  "isBrackish": 0,
  "match_type": "exact",
  "valid_name": "Tripos candelabrus",
  "isFreshwater": 1,
  "isTerrestrial": null,
  "valid_AphiaID": 841193,
  "scientificname": "Tripos candelabrus",
  "unacceptreason": null,
  "valid_authority": "(Ehrenberg) F.Gómez, 2013"
}

The problem now is that WoRMS does not update the modified field, which means that we have no way of knowing the record has changed. The only way to fix this is by manually updating the taxon (we are not going to do that) or by continuously scraping WoRMS for changes. We could also wait for periodic exports from WoRMS but that's not the timescale I have in mind. To be discussed.

wardappeltans commented 5 years ago

Adding a time stamp to the environment flags in WoRMS could solve this. @bart-v?

wardappeltans commented 5 years ago

Adding a time stamp to the environment flags in WoRMS can solve this @bart-v?


From: Pieter Provoost notifications@github.com Sent: Monday, November 12, 2018 9:32:19 PM To: iobis/obis-issues Cc: Appeltans, Ward; Mention Subject: Re: [iobis/obis-issues] Marine species reported as "non-marine" is actually not-accepted (#83)

@auspexhttps://github.com/auspex @wardappeltanshttps://github.com/wardappeltans @bart-vhttps://github.com/bart-v The problem is not with the unaccepted status but with the environment flags of the accepted taxon. These must have been fixed by WoRMS very recently, because this is this is what was fetched from the WoRMS API (not marine, not brackish):

{ "url": "http://www.marinespecies.org/aphia.php?p=taxdetails&id=841193", "lsid": "urn:lsid:marinespecies.org:taxname:841193", "rank": "Species", "class": "Dinophyceae", "genus": "Tripos", "order": "Gonyaulacales", "family": "Ceratiaceae", "phylum": "Myzozoa", "status": "accepted", "AphiaID": 841193, "kingdom": "Chromista", "citation": "Guiry, M.D. & Guiry, G.M. (2018). AlgaeBase. World-wide electronic publication, National University of Ireland, Galway (taxonomic information republished from AlgaeBase with permission of M.D. Guiry). Tripos candelabrus (Ehrenberg) F.Gómez, 2013. Accessed through: World Register of Marine Species at: http://www.marinespecies.org/aphia.php?p=taxdetails&id=841193 on 2018-07-20", "isMarine": 0, "modified": "2015-06-26T14:00:51Z", "authority": "(Ehrenberg) F.Gómez, 2013", "isExtinct": null, "isBrackish": 0, "match_type": "exact", "valid_name": "Tripos candelabrus", "isFreshwater": 1, "isTerrestrial": null, "valid_AphiaID": 841193, "scientificname": "Tripos candelabrus", "unacceptreason": null, "valid_authority": "(Ehrenberg) F.Gómez, 2013" }

The problem now is that WoRMS does not update the modified field, which means that we have no way of knowing the record has changed. The only way to fix this is by manually updating the taxon (we are not going to do that) or by continuously scraping WoRMS for changes. We could also wait for periodic exports from WoRMS but that's not the timescale I have in mind. To be discussed.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/iobis/obis-issues/issues/83#issuecomment-438019011, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AOfln4EMcOv-faoFtfDqgcX3CYur0jHlks5uudrTgaJpZM4YZeFz.

bart-v commented 5 years ago

Sorry, but we cannot change all procedures we have been using for years, just like that. Just refreshing the taxon in OBIS will fix this too.

auspex commented 5 years ago

But how can OBIS know that a taxon needs to be refreshed? Apparently the WoRMS "modified" flag doesn't mean what any new user is going to expect. It's not when the Aphia record was modified but when the taxonomy changed.

bart-v commented 5 years ago

Things are really more complex than this, this requires some discussion, i.e.

As I have said before, relying on the modified flag is not a good idea: you might miss deletes, minor updates, classification changes, etc...

It's best to just harvest WoRMS as a whole. We have decided to make weekly updates available (instead of monthly)