Data Template - April 22

[neil-ices-dk]

@cmspinto to test the idea of an additional header record (non-mandatory) which would allow multiple surveys to be submitted as one file (rather than separate files for each survey). This would mainly affect the backend system - however needs to investigate a little further.

[cmspinto]

This is concluded and the new template is now finished and in available here: https://www.ices.dk/data/Documents/Cetaceans/Cetaceans_data_template.zip It was possible to allow the submissions of multiple surveys in one file and still be compatible with the previous format. So the inclusion of a root header when the file has more than a survey is optional.

We should consider in the next revision of the format to pass some of the fields in the Identifier record to the file header and make the file header mandatory record.

[NikiClear]

@cmspinto @pcrjoana @neil-ices-dk @NikkiTaylorJNCC

While testing the new template by re-uploading the SCANS data, I'm getting an error message (similar to what I was getting before). The data seems to have passed all verification, except the overwrite warning but I can't seem to proceed to upload. Here is a screen grab of what I get when I click 'Import cetaceans data to the database and overwrite the file in the database' button.

[NikiClear]

Hi Carlos,

This is great news!!

I've been testing with a re-upload of SCANS, and the process worked perfectly. However,

While trying to re-submit the SCANS data, I'm getting an error message (similar to what I was getting before). The data seems to have passed all verification, except the overwrite warning but I can't seem to proceed to upload. Here is a screen grab of what I get when I click 'Import cetaceans data to the database and overwrite the file in the database' button.

[image]https://user-images.githubusercontent.com/87769388/168302751-e6b90cd1-1df3-48d2-a77f-a65ced7b7e1d.png

I think I had this error before. I was wondering if you could help me, and test to make sure this won't happen to others who try to upload? (I've also added this issue to GitHub - https://github.com/ices-tools-dev/cetaceans/issues/257)

Thanks, Niki

---

From: Carlos Manuel Silva Pinto @.> Sent: 05 May 2022 11:49 AM To: ices-tools-dev/cetaceans @.> Cc: Niki Clear @.>; Assign @.> Subject: Re: [ices-tools-dev/cetaceans] Data Template - April 22 (Issue #257)

CAUTION: Please remember your Cyber Security training. This email originated from outside the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.

This is concluded and the new template is now finished. It was possible to allow the submissions of multiple surveys in one file and still be compatible with the previous format. So the inclusion of a root header when the file has more than a survey is optional.

We should consider in the next revision of the format to pass some of the fields in the Identifier record to the file header and make the file header mandatory record.

— Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fices-tools-dev%2Fcetaceans%2Fissues%2F257%23issuecomment-1118414570&data=05%7C01%7CNiki.Clear%40jncc.gov.uk%7C7dd0e9de9f38409d107308da2e84fc4e%7C444ee4e8b2fd491d8c318b0508370a6b%7C0%7C0%7C637873445948213302%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=q33QWBg5ffS47YU9UzDI%2F28GMmKv15c83F8CX6MKfts%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAU5UCLEOEYETVOR2DBS5HG3VIORU5ANCNFSM5UBVAWXA&data=05%7C01%7CNiki.Clear%40jncc.gov.uk%7C7dd0e9de9f38409d107308da2e84fc4e%7C444ee4e8b2fd491d8c318b0508370a6b%7C0%7C0%7C637873445948213302%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=hikvVin2GX0mHuxf2SWVaWZCT93rjWTcdTMGH3f5RNU%3D&reserved=0. You are receiving this because you were assigned.Message ID: @.***>

---

JNCC's response to the COVID-19 outbreak is focussed on protecting our people and partners to minimise the potential for the virus to spread. All staff are working from home and we are adhering to the Government’s advice on social distancing and travel restrictions. We are also working with partners to ensure that the projects we support are compliant with the latest Government guidance, including the introduction of restrictions on fieldwork. (See https://jncc.gov.uk/our-work/surveillance-schemes/) Our staff continue to be available for business as usual and will respond to enquiries as promptly as possible, but there may be delays. We ask for your understanding and patience at this time.

For information on how we handle personal data please see our Privacy Notice at https://jncc.gov.uk/privacy

This email and any attachments, is intended for the named recipient(s) only. If you are not the named recipient then any copying, distribution, storage or other use of the information contained in them is strictly prohibited. In this case, please inform the sender straight away then destroy the email and any linked files.

JNCC may have to make this message, and any reply to it, public if asked to under the Freedom of Information Act, data protection legislation or for litigation. If you have a Freedom of Information/Environmental Information request please refer to our website page.

This message has been checked for all known viruses by JNCC through the MessageLabs Virus Control Centre however we can accept no responsibility once it has left our systems. The recipient should check any attachment before opening it.

JNCC Support Co. registered in England and Wales, Company No. 05380206. Registered Office: Monkstone House, City Road, Peterborough, Cambridgeshire PE1 1JY. https://jncc.gov.uk/

[cmspinto]

Hi Niki,

Can you send me the file or what is the SessionID?

[NikiClear]

Hi,

Session ID is 1517https://cetaceans.ices.dk/manage/ScreenResult/1517

I've also had an email from Lucy saying she's also had the 'Developer Mode' error today.

Thanks, Niki

---

From: Carlos Manuel Silva Pinto @.> Sent: 17 May 2022 1:56 PM To: ices-tools-dev/cetaceans @.> Cc: Niki Clear @.>; Assign @.> Subject: Re: [ices-tools-dev/cetaceans] Data Template - April 22 (Issue #257)

CAUTION: Please remember your Cyber Security training. This email originated from outside the organisation. Do not click links or open attachments unless you recognise the sender and know the content is safe.

Hi Niki,

Can you send me the file or what is the SessionID?

— Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fices-tools-dev%2Fcetaceans%2Fissues%2F257%23issuecomment-1128835346&data=05%7C01%7CNiki.Clear%40jncc.gov.uk%7Cedcd09f6610c4c0c906f08da3804b96e%7C444ee4e8b2fd491d8c318b0508370a6b%7C0%7C0%7C637883890181491391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=NjyTrx7N79oEk6AHZ2yqEwgA4yfvOGeracVFH09%2FMFw%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAU5UCLE6CR3OOHKBWA3UORDVKOJRNANCNFSM5UBVAWXA&data=05%7C01%7CNiki.Clear%40jncc.gov.uk%7Cedcd09f6610c4c0c906f08da3804b96e%7C444ee4e8b2fd491d8c318b0508370a6b%7C0%7C0%7C637883890181491391%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4fopasccvt2cBmV8L3fKVKsZ%2BlFKgjxNvQuVRR8bSIs%3D&reserved=0. You are receiving this because you were assigned.Message ID: @.***>

---

JNCC's response to the COVID-19 outbreak is focussed on protecting our people and partners to minimise the potential for the virus to spread. All staff are working from home and we are adhering to the Government’s advice on social distancing and travel restrictions. We are also working with partners to ensure that the projects we support are compliant with the latest Government guidance, including the introduction of restrictions on fieldwork. (See https://jncc.gov.uk/our-work/surveillance-schemes/) Our staff continue to be available for business as usual and will respond to enquiries as promptly as possible, but there may be delays. We ask for your understanding and patience at this time.

For information on how we handle personal data please see our Privacy Notice at https://jncc.gov.uk/privacy

This email and any attachments, is intended for the named recipient(s) only. If you are not the named recipient then any copying, distribution, storage or other use of the information contained in them is strictly prohibited. In this case, please inform the sender straight away then destroy the email and any linked files.

JNCC may have to make this message, and any reply to it, public if asked to under the Freedom of Information Act, data protection legislation or for litigation. If you have a Freedom of Information/Environmental Information request please refer to our website page.

This message has been checked for all known viruses by JNCC through the MessageLabs Virus Control Centre however we can accept no responsibility once it has left our systems. The recipient should check any attachment before opening it.

JNCC Support Co. registered in England and Wales, Company No. 05380206. Registered Office: Monkstone House, City Road, Peterborough, Cambridgeshire PE1 1JY. https://jncc.gov.uk/

[cmspinto]

Found the bug, but I haven't tried because I don't want to resubmit your data, could I ask you to try now?

[NikiClear]

@cmspinto

I've tried the resubmission again this morning. The previous error has been sorted, thank you. But I've got a new error now. When I try re-upload I'm brought back to the manage dashboard with an error saying the data couldn't be uploaded (see below)

I used the metadata URL - https://gis.ices.dk/geonetwork/srv/eng/catalog.search#/metadata/04cdd959-d36b-4f6e-b3e7-622268d719b1

Any ideas what's causing this?