ices-eg / WGJCDP-Cetaceans

Joint Cetaceans Data Portal (development)
Creative Commons Zero v1.0 Universal
4 stars 0 forks source link

Suggested Check: number of calves < group size #250

Closed neil-ices-dk closed 2 years ago

neil-ices-dk commented 2 years ago

From Lucy

Additional comments:

Validation rules:

Number of calves cannot be greater than the group size If you did employ species being flagged if recorded in certain reasons, how would the region be defined? Outside of known range; outside of known range but with a large buffer area; even more broad than that? SightingAngle - I know it is limited to 3 digits, and ignore me if aerial surveys record this differently, but this should only be between 0 - 359 RadialDistance - maybe consider flagging sightings past a certain distance PerpendicularDistance - maybe consider flagging sightings past a certain distance

NikiClear commented 2 years ago

Submitted during SG meeting:

@NikkiTaylorJNCC @neil-ices-dk @cmspinto @pcrjoana

neil-ices-dk commented 2 years ago

Approve now:

  1. Number of calves cannot be greater than the group size;

For WGJCDP

  1. If you did employ species being flagged if recorded in certain reasons, how would the region be defined?
  2. Species not in the database should be flagged as a warning;
pcrjoana commented 2 years ago

@pcrjoana to implement the check 1.

neil-ices-dk commented 2 years ago

suggested by NikiC: The screening should check for duplication in the ID fields of each table within a survey

neil-ices-dk commented 2 years ago

@pcrjoana to implement 'duplication in the ID fields' also to check the previous check on species in a specific region has been implemented

pcrjoana commented 2 years ago

@NikiClear I need a refresh here. What do you mean by "duplication in the ID fields"?

NikiClear commented 2 years ago

@NikiClear I need a refresh here. What do you mean by "duplication in the ID fields"?

Hi @pcrjoana - I believe the SurveyID EffortID and SightingID fields should be unique with no duplication within the same survey dataset

neil-ices-dk commented 2 years ago

@cmspinto to pick this up - thank you

cmspinto commented 2 years ago

All three checks have been added to the format: SightingID must be unique within a file: http://[datsu.ices.dk/web/chkDetails.aspx?DatasetID=149&RecordID=110&groupFilterID=1951](http://datsu.ices.dk/web/chkDetails.aspx?DatasetID=149&RecordID=110&groupFilterID=1951)

SurveyID must be unique within a file: http://[datsu.ices.dk/web/chkDetails.aspx?DatasetID=149&RecordID=108&groupFilterID=1949](http://datsu.ices.dk/web/chkDetails.aspx?DatasetID=149&RecordID=108&groupFilterID=1949)

EffortID must be unique within a file: http://[datsu.ices.dk/web/chkDetails.aspx?DatasetID=149&RecordID=108&groupFilterID=1950](http://datsu.ices.dk/web/chkDetails.aspx?DatasetID=149&RecordID=108&groupFilterID=1950)

Could this be checked by putting a file where one of them is repeated?

neil-ices-dk commented 2 years ago

Could this be checked by putting a file where one of them is repeated?

assume you'd like @NikiClear to check that?

NikiClear commented 2 years ago

Duplication and number of animals checks have been tested and it seems to be working as it should - https://cetaceans.ices.dk/manage/ScreenResult/1607

Tested the same dataset without any duplication or problems, but I got an error suggesting duplication in the SurveyID where it shouldn't be - there is only one record in the upload file. https://cetaceans.ices.dk/manage/ScreenResult/1609 SCANS_BlockP_TestingNoDup.zip

Osanna123 commented 2 years ago

SurveyID check edited. Should work now

NikiClear commented 2 years ago

SurveyID check edited. Should work now Checked and it's working now, thanks!

pcrjoana commented 2 years ago

All the checks seem to be in place. Can we close this issue?

NikkiTaylorJNCC commented 2 years ago

All the checks seem to be in place. Can we close this issue?

Hi @pcrjoana !! If you're happy they are covered then absolutely, we can close it.