Review the draft Data Flow for Acoustic data

[neil-ices-dk]

Part of a wider QA of advice effort, see example here

Currently, the plan is to have the acoustic dataflow published January 2021

[HjalteParner]

@CiaranOD et all, https://community.ices.dk/Publications/dataflows/Shared%20Documents/4.%202020_Acoustic/DataFlowAcoustic.pdf is the link to the Acoustic Data Flow going for Accreditation and which should be reviewed by the Acoustic Governance Group. Please have a look and lets dissused this data flow in upcoming Acoustic Governance Meeting 10th of December.

Any comments would be welcomed in this thread as well before the meeting!

[CiaranOD]

@HjalteParner - had a look over this and you have captured the process concisely. I think it would be a nice to send this link out pre-the meeting as a means to get attendees into the 'zone' in advance. Plus they will have their comments ready for input at the meeting.

[HjalteParner]

@CiaranOD yes I agree. Will you send it out or should we get Karolina to do it for us?

[CiaranOD]

@HjalteParner . I would ask Karolina- to be consistent.

[HjalteParner]

@CiaranOD sounds good. I also intend to add some other issues :-) to the backlog tomorrow!

[SMLusseau]

This is really nice and clear! Will this then be beefed out with the specifics of checks etc that should be undertaken in the different steps?

[CiaranOD]

We didn't get around to reviewing this document as I got a bit carried away with the PBI discussion. My apologies to Joana !

[CiaranOD]

We still need to review this Data Flow document to ensure it outlines correctly our workflow so this can be published. I'll circulate an email to the group with a deadline for comments.

https://community.ices.dk/Publications/dataflows/Shared%20Documents/4.%202020_Acoustic/DataFlowAcoustic.pdf

[neil-ices-dk]

We still need to review this Data Flow document to ensure it outlines correctly our workflow so this can be published. I'll circulate an email to the group with a deadline for comments.

https://community.ices.dk/Publications/dataflows/Shared%20Documents/4.%202020_Acoustic/DataFlowAcoustic.pdf

and just for context, this dataflow schematic would sit within a publication denoting linkages to other dataflows, the context of the dataflow etc. just re-pinning the completed example for reference

[CiaranOD]

Hi Neil,

I've noted this with the group for review and a deadline for comments. I think you were cc'd ?

Regards,

Ciaran

-------- Original message -------- From: Neil Holdsworth notifications@github.com Date: 15/12/2020 20:58 (GMT+00:00) To: ices-eg/wg_WGACOUSTICGOV wg_WGACOUSTICGOV@noreply.github.com Cc: Ciaran O'Donnell Ciaran.O'Donnell@Marine.ie, Mention mention@noreply.github.com Subject: Re: [ices-eg/wg_WGACOUSTICGOV] Review the draft Data Flow for Acoustic data (#5)

We still need to review this Data Flow document to ensure it outlines correctly our workflow so this can be published. I'll circulate an email to the group with a deadline for comments.

https://community.ices.dk/Publications/dataflows/Shared%20Documents/4.%202020_Acoustic/DataFlowAcoustic.pdf

and just for context, this dataflow schematic would sit within a publication denoting linkages to other dataflows, the context of the dataflow etc. just re-pinning the completed examplehttp://ices.dk/sites/pub/Publication%20Reports/Data%20Visualization/ICES%20Data%20Flow%20Schematics_No.1-VMS-NEAFC.pdf for reference

- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ices-eg/wg_WGACOUSTICGOV/issues/5#issuecomment-745562821, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACIG7CREO3RRHWILEJWV47DSU7EV5ANCNFSM4RLU7F4Q.

[jmjech]

Hi Ciaran, I'm writing my comments here. Let me know if you want me to send via e-mail as well. I'm signed in under my username: jmjech.

1. It seems there should be a decision node where the flow bifurcates from the "Data request" box (in Definition of data required) to either the survey design decision node (which I think should be a procedure box rather than a decision node) or all the way down to "the advice is published" end of workflow. I'm not sure why that flow goes all the way there? In any event, it seems that if there is a data request, there are three options: 1) data exist and just need to be packaged for the new request, 2) data exist, but may require additional processing and packaging, or 3) data do not exist, so new data need to be collected.
2. Data collection, survey design decision node. I think this should be a procedure box rather than a decision node. It seems the decision has already been made to design a new survey.
3. Data collection, 2nd procedure box. I suggest revising the 2nd sentence to "Trawl catches are analyzed for ..."
4. Estimation: abundance indices grouping, upper left communication box. It seems that the format check has already been done in the Data submission grouping. Maybe a rewording of the text to "Data are not sufficient for survey group needs" The flow then actually goes all the way back to the survey design diamond in the data collection grouping.
5. Estimation: abundance indices grouping. Why does the data flow break off to the Assessment grouping before the survey group quality check? It seems that flow should be done after the quality check. This then would eliminate the need for the communication box in the upper left of the Assessment grouping.
6. Assessment grouping. It seems that the quality check by the assessment WGs is done not for compliance, but to decide whether the data can be used in an assessment. If the data can not, then the flow should go all the way back to the data collection grouping and survey design.
7. As a general comment, it seems there are quality checks for the data and then quality checks for the usefulness of the data. The data that are collected should be quality checked for accuracy and precision and format. Once they are accepted and input to the database, then it is up to the survey and assessment (and ecosystem) groups to decide if the data are useful. If not, then it is not a problem with the data per se, but with the type of data collected. I think this should be the main feedback loop from the assessment/ecosystem groups to the data collecting groups. We discussed this quite a bit at WKREO (led by Sven Kupschus), in that there didn't seem to be a good mechanism for comments and data needs to get communicated back to the people who actually collect the data.

Best, Mike Jech

[SMLusseau]

I think Mike has spotted some good issues with the data flow in his review, much of which many of us might not have spotted as we are perhaps too familiar with the processes described to see where they may not be described in a clear way. I will add my own comments using the same numbers as Mike:

1. I also wondered about this direct link to the “advice is published” box at the end, might need to specify what this link means and its direction. I also agree with Mike that there is a step in between the request for data and the design of the survey where the other two outcomes are related to evaluating existing data sources. I am thinking we are here talking about re-scrutinising acoustic survey data for species that were not the original target for example.
2. I agree with this
3. I think that is how it reads now in the version I looked at?
4. The diagram reflects the work flow as it is, but I can see how it might be a little confusing as it does indeed read as if the same check is done twice and maybe it needs to be worded better. There is an automatic format check when the data is uploaded to the database and at this point users will be asked to correct and re-upload to proceed. The data that has successfully passed these automated checks then go through a second round of checks in the post cruise meetings and again this scrutiny can lead to data being corrected and resubmitted. Some examples of this is in my answer to an earlier post about reason why data gets resubmitted during post cruise meetings. Typically they are caused by inconsistencies in the uploaded data amongst the different participating nations in the survey. A rewording may indeed help here, but I don’t think the arrow need to go back to survey design although a feedback loop to improving the survey manual in regards to data delivery requirements may be useful.
5. I agree the line from between data extraction and survey group review to assessment is confusing. There should maybe be an arrow to clarify that this is a one way street where the survey group can be contacted by the assessment group to review survey data if the assessment group finds inconsistencies when scrutinizing the data for use.
6. I agree with this to a large extent but also know that as we are moving into this system of increased checks and balances on the survey results we still find some errors or inconsistencies unearthed by the assessment groups in older data that warrants an investigation by the survey group.
7. I wholly agree with this comment and hope this is the direction we are travelling in with this data flow document for example, and other initiatives like the common database and increased focus on manuals being up to date etc is helping us get there too. Between HAWG and WGIPS we have also tried to set up a survey summary reporting system between the groups to increase the communication both ways but so far it unfortunately seem to become a box checking exercise for most.

[CiaranOD]

No feedback as yet on this review but thank you for your input

@neil-ices-dk @HjalteParner

[neil-ices-dk]

@pcrjoana please take a look at the comments above related to the draft dataflow and discuss with Hjalte - thanks

[jmjech]

The revised data flow is an improvement. I have two main comments.

1. It seems this flow jumps from a request to a brand new survey. I think there should be some discussion box where there is a dialog between the survey group and the benchmark/assessment group to see if an entirely new survey or "just" a change in collection protocol is required.
2. I don't understand the long line on the right from data request at the top all the way to advice is published.

[pcrjoana]

@jmjech

1. The schematic tries to encompass the basic structure of the data flow. Maybe we can change the wording in the decision box, so it reflects better what happens. Are surveys exactly the same each year, or are decisions made for each year's survey, even if the overall structure is repeated over the years? The decision box was based on the assumption that the latter would hold true.
2. The long line is to reflect that this is a process that happens each year.

[pcrjoana]

New version: http://community.ices.dk/Publications/dataflows/Shared%20Documents/4.%202020_Acoustic/DataFlowAcoustic.pdf

[CiaranOD]

Hi Joana,

After review of the document, one issue was highlighted for editing:

Data Collection; The text reads: The survey group plans a survey based on the target species and stock assessment requirements.

This should read: The survey group evaluates the need for a survey based on a review of the historic data and stock assessment requirements.

The arrow should fork at the location where a new survey is needed (Yes/No).

Thanks and apologies for the delay!

[HjalteParner]

@pcrjoana did you make the last changes and publish the acoustic data flow?

[pcrjoana]

Hello everyone. I'm sorry for the late reply. The changes have been implemented, but don't want to add a "no survey needed" box before I get a clarification from @CiaranOD. Where should the "no survey needed" box connect to? If the dataflow stops there if there's no need for a survey, I think it's OK to leave as is, but please let me know how you prefer it. :-) Data flow schematic

[CiaranOD]

Hi Joana,

Yes, Dataflow stops at 'no survey needed'. No further action is required.

Cheers,

Ciaran

[pcrjoana]

Alright. Then I think it's OK to leave the dataflow as it is in the link above. Do you agree with that, @CiaranOD?

[CiaranOD]

I agree with that @pcrjoana :-)

[pcrjoana]

Great! I will pass it on to Neil then, and hopefully the data schematic will be published still this year. :-)

[HjalteParner]

@pcrjoana, what's the status on publishing the acoustic data flow?

[neil-ices-dk]

@HjalteParner content wise it is now fine (couple of small edits were needed); Joana is in the process of reformatting the diagram as it was one continuous flow and it needs to be broken into A4 chunks for publication. Once Joana has done that, i will publish straight away

[neil-ices-dk]

@CiaranOD we now have an agreed acoustic dataflow, sorry it has taken a while to get this far. We now need some informational context for the publication - specifically some metadata which you can see in the example from the VMS dataflow (link and image)

https://doi.org/10.17895/ices.advice.7670

So table 1 is really the high level processes that this dataflow feeds, we can pretty much fill this and it can be as generic as 'ICES Assessment Working Groups', however if you have a list of specific assessment working groups directly using the acoustic outputs in their assessments, that would be preferable; the 2nd part of table 1 is denoting links to other dataflows, where the relationship can be either acoustic requires dataflow A or that dataflow A is required by acoustic. If there are no dependencies, then just note that.

table 2 is specific to the advice process and denotes where the dataflow directly relates to a recurring advice product. I imagine that if table 1 is filled in to show fisheries assessment WG's using the outputs for their work, then this would imply they are linked to Fisheries Overviews. We can also confirm this with the FO team once the WGACOUSTICGOV have had a view on this.

Hope that makes sense, and call me if you need more guidance. Thanks, Neil

[neil-ices-dk]

@CiaranOD just a reminder on this one - thanks!

[CiaranOD]

Thanks @neil-ices-dk Neil, we'll have a look at it and get back in due course.

[pcrjoana]

Hello @CiaranOD any news on this one?

[CiaranOD]

@pcrjoana in short no. If you want to have a bash at it we can review. Thanks.

[pcrjoana]

@CiaranOD can we review this and get the tables sorted? If needed we can have a meeting to get it done. It would be good if this could be done before Christmas, so that we could publish this dataflow first thing in 2023.

[neil-ices-dk]

@HjalteParner can you provide the dependency table asap as we will go ahead and publish this version; the governance group can then work on the next version. thanks, Neil

[pcrjoana]

@HjalteParner have you provided the dependency table to @neil-ices-dk yet? If not, can you do that latest by Monday? I'm putting the schematic together for publication and that's an essential piece!

[HjalteParner]

@pcrjoana the acoustic trawl portal governance group is currently parked as no chair have been found. This task will as such need to be parked as well or you need to reach out to the 4 survey coordinating groups WGIPS, WGBIFS, WGACEGG and WGIDEEPS

[neil-ices-dk]

@HjalteParner we cannot wait any longer (3 years is long enough) and we need to publish a first version, even if it is not complete. Therefore, please do your best to complete the dependency table, and we can then bring it back to the group once a new chair has been found. Thanks

[HjalteParner]

@odontaster?

[pcrjoana]

@HjalteParner and @odontaster how are we on the dependency table? I'd like to have it by Wednesday this week, if possible.

[pcrjoana]

@HjalteParner and @odontaster I made this as easy and painless for you as possible - please go into the link below and fill in the "costumers" section of the meta-information table (section 1), and also fill in the table regarding linked data flows, if there are any. It shouldn't take you more than 10 minutes. Please do this before Easter.

https://community.ices.dk/Publications/dataflows/Shared%20Documents/4.%202020_Acoustic/ICES%20Data%20Flow%20Schematics_Acoustic.docx

[HjalteParner]

@pcrjoana can you please provide a link here to the published version?

[neil-ices-dk]

@HjalteParner it is not published yet https://ices-library.figshare.com/pubtype_publications_data-flow-schem

[odontaster]

https://doi.org/10.17895/ices.pub.23904465.v1

Linked also from Acoustic main page https://www.ices.dk/data/data-portals/Pages/acoustic.aspx