QC: Feedback and reported issues

[NikiClear]

Reported issues from data providers

• When uploading our data, it passes the screening with no errors, but when we select 'import cetaceans data to the database' an error message occurs

• I emailed accessions@ices.dk several times to request the R script, but have never had a reply so we are still waiting for this

• We have emailed Joana regarding the missing vessels, but again no reply and the vessels are still not in the database - these were requested back in January.

• The Data Standard says the abstract (in the identifiers table) can be a maximum of 500 words but the data rules on the portal limit it to 500 characters @NikiClear to update the data standard

• Duplicate sighting status is mandatory even for single platform surveys, although this is supposed to be conditional for double platform surveys

• Data rights holder isn’t actually free text as it states in the Data Standard, it is actually an EDMO code

• Milling/non - directional behaviour and Globicephala and Balaenopteridae AphiaIDs: for these, you have to reselect every one from the dropdown menu even though they’re in exactly the same format and this is very laborious. Can this be looked into please so it automatically selects them from the dropdown when entered, like the other values do?

• Survey with no sightings has failed validation and not able to upload, the system should accept surveys with no sighting records.

[neil-ices-dk]

a. This was caused by an internal call to a service as there were new species to be added to the DB from Lucy’s data; we didn’t catch this in testing as we worked with a file that had species that were already known to the database. NOW FIXED

b. R script for bulk upload: we only have it for another data portal, JNCC was tasked with looking for resources to do this for Cetaceans

c. Vessels; we are in process but this takes time as we need to get NOAA to approve, and they often do this in bulk at periodic intervals.

d. Word/Character limit on Survey Abstract field (identifiers); Joana will change in data format (DATSU) to 8 000 characters (we can’t limit by words); NOW FIXED

e. Duplicate sighting – conditional mandatory; We think the data standard field should be ‘mandatory’ (as it is a vocab) and as it is in the template (the sighting number is conditional optional depending on the answer to duplicatesighting status).

f. DataRightsHolder – we think we agreed to the EDMO, but we can’t find any documentation on that, can we confirm in next sprint

g. Milling/non directional behaviour and Globicephala and Balaenopteridae AphiaIDs; @pcrjoana to have a dedicated session with Lucy to understand what she wants (not clear from description)

h. Survey with no sightings; this was not anticipated as we agreed to have the whole sightings record as mandatory, not optional. To discuss at next sprint, it can be changed to optional if this scenario should be included.

[NikkiTaylorJNCC]

a. This was caused by an internal call to a service as there were new species to be added to the DB from Lucy’s data; we didn’t catch this in testing as we worked with a file that had species that were already known to the database. NOW FIXED

b. R script for bulk upload: we only have it for another data portal, JNCC was tasked with looking for resources to do this for Cetaceans

c. Vessels; we are in process but this takes time as we need to get NOAA to approve, and they often do this in bulk at periodic intervals.

d. Word/Character limit on Survey Abstract field (identifiers); Joana will change in data format (DATSU) to 8 000 characters (we can’t limit by words); NOW FIXED

e. Duplicate sighting – conditional mandatory; We think the data standard field should be ‘mandatory’ (as it is a vocab) and as it is in the template (the sighting number is conditional optional depending on the answer to duplicatesighting status).

f. DataRightsHolder – we think we agreed to the EDMO, but we can’t find any documentation on that, can we confirm in next sprint

g. Milling/non directional behaviour and Globicephala and Balaenopteridae AphiaIDs; @pcrjoana to have a dedicated session with Lucy to understand what she wants (not clear from description)

h. Survey with no sightings; this was not anticipated as we agreed to have the whole sightings record as mandatory, not optional. To discuss at next sprint, it can be changed to optional if this scenario should be included.

RESPONSE: b - R script- this had completely left my brain but now you mention it...I remember. We'll take this internally and see if we can convince our coders to take it on!

c. vessels - possibly a small communication error here, I thought Lucy was still in the process of sorting the details with Joana, rather than it be a case of waiting for NOAA to add them. If the latter is the case, not much we can do, although if there is a way to speed that up it would be fab as Orca are poised to get their data in.

e. duplicate sighting would be best as conditional, based on selecting double platform methodology. Otherwise all non double platform methods will have to complete this extra step with zeros.

f. datarightsholder - definitely free text! The datacustodian is the only one required to have an EDMO. This is to reduce the burden on data rights holders who have very little to do with their data, and commission it for e.g. offshore industry. Additional requirements to get an EDMO could be offputting and result in data not being submitted.

h I think we will have to make it optional as there will be some surveys with no sightings.

[pcrjoana]

@NikkiTaylorJNCC regarding e: if it's not a double platform, they can just enter that it is not a duplicate sighting and fill the record without any issue. If in DuplicateSightingStatus "No" is entered, DuplicateSightingNumber is not required. Do you still want to have DuplicateSightingStatus as conditional? To me it doesn't make much sense, because if it's not a duplicate, they have the option to enter "No".

[NikkiTaylorJNCC]

@NikkiTaylorJNCC regarding e: if it's not a double platform, they can just enter that it is not a duplicate sighting and fill the record without any issue. If in DuplicateSightingStatus "No" is entered, DuplicateSightingNumber is not required. Do you still want to have DuplicateSightingStatus as conditional? To me it doesn't make much sense, because if it's not a duplicate, they have the option to enter "No".

@pcrjoana we can do either, but my thoughts were that if we make it conditional at the methodology stage, they don't have to worry about the extra step of filling out the duplicatesightingstatus as that has no relevance to non-double platform methodologies. But to be honest, i'm happy with the path of least resistance! Whatever is simpler to do, either make duplicatesightingstatus conditional based on double platform methodologies being selected, or make duplicatesightingstatus mandatory and maybe I need to try and make it clearer that non-double platform methods need to enter 'no' across the board here.

[pcrjoana]

f. - done h. - sightings record is now optional

@NikkiTaylorJNCC regarding e: if it's not a double platform, they can just enter that it is not a duplicate sighting and fill the record without any issue. If in DuplicateSightingStatus "No" is entered, DuplicateSightingNumber is not required. Do you still want to have DuplicateSightingStatus as conditional? To me it doesn't make much sense, because if it's not a duplicate, they have the option to enter "No".

@pcrjoana we can do either, but my thoughts were that if we make it conditional at the methodology stage, they don't have to worry about the extra step of filling out the duplicatesightingstatus as that has no relevance to non-double platform methodologies. But to be honest, i'm happy with the path of least resistance! Whatever is simpler to do, either make duplicatesightingstatus conditional based on double platform methodologies being selected, or make duplicatesightingstatus mandatory and maybe I need to try and make it clearer that non-double platform methods need to enter 'no' across the board here.

I kept it mandatory, and added the guidance that "No" should be filled in when single platform methodologies are used. Path of least resistance. ;-) They only need to fill that field with "No", as the other fields related to duplicate sightings are conditional on the basis of it being "Yes" or "Probable" (this condition is already impletemented).

[neil-ices-dk]

v2.03 of data template live on web addressing the issues above; will wait for next sprint before concluding and closing issue.