How to handled CSR's at ICES - for DIG?

[HjalteParner]

ICES DC is using Cruise Summary Reports - CSR's as the core metadata catelogue for oceanography and acoustic and should problaly expand to other datasets as well - environment as fisheries.

The CSR system is based on ROSCOPs which orginal was manage at ICES but the manegement was years ago taken over by SeaSearch, SeaDataNet and lately SeaDataCloud using controlled vocabularies instead of free text. The ICES Platform vocabulary is still core to the CSR system.

German BSH has been in charge of the CSR system on behalf of SDN/C from the beginnning but lately France IFREMER have taking over this responsibility. ICES is replicating the CSR's system to our own server every nigth.

SDN/C: https://www.seadatanet.org/Metadata/CSR-Cruises BSH: http://seadata.bsh.de/Cgi-csr/retrieve_sdn2/start_sdn2.pl IFREMER: https://csr.seadatanet.org ICES replicate: https://ocean.ices.dk/csr

ICES stil recieves ROSCOPs / CSRs from some member countries (even though only a few per year) which arn't involved in the EC SDC (currently not funded). However as IFREMER are manageging the CSR system currently, we don't want to complicate thing and would like our users to commit their CSRs to SDC and then ICES will make use of the SDC interface when looking up cruise metadata for data quality control.

Do you agree on this approach?

How can we make ICES data (CSR) submitters make use of the SDC system?

And then the obvious question, can we trust in SDC (funded or not) to maintain the CSR system on which we rely on?

@sjurl @neil-ices-dk can we please discuss this at this years DIG?

[HjalteParner]

@sjurl @neil-ices-dk would it be possible to discuss the CSR issue at this year DIG?

[HjalteParner]

I have approached IFREMER on the CSRs status and plans (see below correspondence).

They are not surprisingly finding issues with the C17 vocabulary and old CSR’s as well as new CSRs created at ICES based on data using deprecated multiple hulls platform codes not in C17. I and (@mehdiabbasi) will find a pragmatic solution to this and we won’t be creating CSRs based on data at ICES anymore looking ahead.

---

Dear Hjalte,

I hope you are fine.

Due to the resignation of one of our developpers, we are a little bit late for the developement of the new CSR back-office (the online tool which will allows to create and update CSR entries). The first version of this tool is currently being tested, I hope to move it into production soon and to open it for the ISO 19139 SeaDataNet XML submissions. After that, we will be able to start the development of the second version of the CSR back-office which will includes the CSR submission using web forms.

So, for the moment, we are not ready for the final transfert from BSH to IFREMER and we have to continue to maintain the both BSH and IFREMER systems for a few months.

I was quite busy at the beginning of 2021 with the organisation of the IMDIS conference. That is why I didn't synchronise our database with the BSH database since January. A synchronisation was in progress when you sent me this email, it is now finished and you can find some 2021 CSR entries on the IFREMER web interface. In the coming months, I will synchronise more regularly.

We have made some progress with the ICES CSRs. On April 30th, my IT team captured 23005 CSRs entries using your webservices, but I have some troubles to ingest them in our database because: • some of them makes reference to deprecated C17 terms (ex: 67LT, 26DA, 0699, 74SC, etc). Can you do something about it? If not, I will replace these deprecated terms by ZZ99 - Unknown term and keep the platform name as free text in a dedicated field. • some of them are duplicates with existing V2 entries. For the moment, I identified this problem only with French cruises (IFREMER recently updated all these old entries). I don't know how to handle these cases. Do you have any advice? Here are some examples:

ICESREF BSHREF 199135116 20176264 199635095 19966131 199635101 19966130 199835087 19986261 199835088 19986265 199935041 19997110 199935099 19997119 199935100 19997121 199935104 19997128 199935105 19997130 200235088 20027088 200435131 20047178 200435132 20047179 199435082 20155284 199435083 20155286 199435084 20155288 199435085 20155287

I was expecting to retrieve only "old" CSR entries (V0) using your webservices but I realized that we captured also recent CSRs entries (cruises from 2010 to 2020, ex: ICES 202067002). Is it normal? Is this what you expected?

Thanks in advance for your help

Best regards Vanessa

Le 07/05/2021 à 10:50, Hjalte Parner a écrit : Dear Vanessa,

Hope this mail finds you well.

I was wondering about the status of transferring SDN CSRs from BSH to IFREMER + if you actual manage to capture ICES CSRs though our web service and if not the status here?

If I query the web interfaces at IFREMER and BSH, I get 46242 and 62725 CSR records respectively and it seems like no 2021 CSR records exist in the IFREMER system?

What’s your plan? We are going to discuss how ICES should use the CSR system i.e. can we stop our CSR system and rely on the BSH CSR system or would IFREMER CSR system soon be in a stage where we could rely on this in our data quality control procedures?

Best regards,

Hjalte

[HjalteParner]

I got an update from IFREMER (Michèle Fichaut) on some of the questions raised by myself.

I would like to assure you that IFREMER, which invested a lot of its own funds for the development of the new database and the CSR front-office and back-office, will maintain it on the long term.

Then concerning the other question: How can we make ICES data (CSR) submitters make use of the SDC system? The new online form will be opened in the first of 2022 to replace the one that was maintained by BSH. This of course could be used by any external users wishing to submit CSR to SeaDataNet system, and will be also maintained on the long-term.